Shibhansh Dohare

I am a Ph.D. student in the RLAI lab at the University of Alberta, advised by Dr. Richard Sutton and Dr. Rupam Mahmood. I did my undergrad at IIT Kanpur, majoring in Computer Science and Engineering.

My long-term research goal is to understand the working of our minds. Specifically, to find the fundamental computational principles that give rise to the mind. In pursuit of this goal, I'm working on various aspects of continual learning, deep learning, and reinforcement learning.

Besides research, I love climbing, social dancing, trying new food, running, playing board games, and the mountains.

Email  /  CV  /  Google Scholar  /  Twitter  /  Github

profile photo
     [Sept 2023]      We published a paper on policy collapse in EWLR 2023.
     [Aug 2023]      Our paper, Loss of Plasticity in Deep Continual Learning, is finally available on arxiv.
     [Aug 2022]      I shared a keynote keynote at CoLLAs with Rich on Maintaining Plasticity in Deep Continual Learning.

My favorite papers are highlighted.

Overcoming Policy Collapse in Deep Reinforcement Learning
Shibhansh Dohare, Qingfeng Lan, A. Rupam Mahmood
EWRL 2023

We show that popular deep RL algorithms, like PPO, do not scale with experience. Their performance gets worse over time. We look deeper into this problem and provide simple solutions to reduce performance degradation.

Loss of Plasticity in Deep Continual Learning
Shibhansh Dohare, J Fernando Hernandez-Garcia, Parash Rahman, Richard S. Sutton, A. Rupam Mahmood
Paper | Code | Talk | Slides

Provide the first direct demonstration of the loss of plasticity faced by deep learning methods in continual learning problems.

We propose a new algorithm, continual backpropagation, that fully maintains plasticity. Continual backpropagation re-initializes a small fraction of less-used units and performs a gradient descent step at each update.

Automatic Noise Filtering with Dynamic Sparse Training in Deep Reinforcement Learning
Bram Grooten, Ghada Sokar, Shibhansh Dohare, Elena Mocanu, Matthew E. Taylor, Mykola Pechenizkiy, Decebal Constantin Mocanu,
AAMAS 2023

We show that standard Deep RL algorithms fail when the input contains noisy features. Dynamic sparse training successfully filters through the noisy features and performs well.

Gamma-Nets: Generalizing Value Estimation over Timescale
Craig Sherstan, Shibhansh Dohare, James MacGlashan, Johannes Günther, Patrick M. Pilarski,
AAAI, Oral Presentation, 2020

We present Gamma-nets, a method for generalizing value function estimation over timescale.

Unsupervised Semantic Abstractive Summarization
Shibhansh Dohare, Vivek Gupta, Harish Karnick,
ACL, Student Research Workshop, 2018

A novel algorithm for abstractive text summarization based on Abstract Meaning Representation.

Website design credit goes to Jon Barron.