mlsanityData to 7 Oct 2026

Self-Imitation Learning for Robot Tasks with Sparse and Delayed Rewards

arXiv 2010.06962 ↗First posted 14 Oct 2020cs.LG cs.AIcited by 4

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

The application of reinforcement learning (RL) in robotic control is still limited in the environments with sparse and delayed rewards. In this paper, we propose a practical self-imitation learning method named Self-Imitation Learning with Constant Reward (SILCR). Instead of requiring hand-defined immediate rewards from environments, our method assigns the immediate rewards at each timestep with constant values according to their final episodic rewards. In this way, even if the dense rewards from environments are unavailable, every action taken by the agents would be guided properly. We demonstrate the effectiveness of our method in some challenging continuous robotics control tasks in MuJoCo simulation and the results show that our method significantly outperforms the alternative methods in tasks with sparse and delayed rewards. Even compared with alternatives with dense rewards available, our method achieves competitive performance. The ablation experiments also show the stability and reproducibility of our method.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 12 of its 22 references that are in the map. First posted on arXiv 14 Oct 2020.

Builds on

12 of 22 references.

Closest ancestors · 1

Same direction, newest first

  1. 2018Learning Self-Imitating Diverse Policies25 May

Other references · 11

  1. 2016OpenAI Gymcited by 1,684
  2. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  3. 2017Hindsight Experience Replaycited by 1,096
  4. 2018Exploration by Random Network Distillationcited by 966
  5. 2019Learning agile and dynamic motor skills for legged robotscited by 402
  6. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 361
  7. 2020The Ingredients of Real-World Robotic Reinforcement Learningcited by 111
  8. 2018Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Costcited by 79
  9. 2018A Large-Scale Study on Regularization and Normalization in GANscited by 26
  10. 2020Scalable Multi-Task Imitation Learning with Autonomous Improvementcited by 21
  11. 2019Learning Hierarchical Control for Robust In-Hand Manipulationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 2; 2026: 1.

4 papers.

  1. 2023A Survey of Temporal Credit Assignment in Deep Reinforcement Learningcited by 56
  2. 2025Attention-Based Reward Shaping for Sparse and Delayed Rewardscited by 5
  3. 2025Towards a Reward-Free Reinforcement Learning Framework for Vehicle Controlcited by 0
  4. 2026Temporal Self-Imitation Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020Episodic Self-Imitation Learning with Hindsight
  2. 2019Reinforced Imitation in Heterogeneous Action Space
  3. 2022STIR2: Reward Relabelling for combined Reinforcement and Imitation Learning on sparse-reward tasks
  4. 2026Generalizable Dense Reward for Long-Horizon Robotic Tasks
  5. 2020Learning Sparse Rewarded Tasks from Sub-Optimal Demonstrations
  6. 2022Self-Imitation Learning from Demonstrations
  7. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning
  8. 2019Watch, Try, Learn: Meta-Learning from Demonstrations and Reward