mlsanityData to 7 Oct 2026

Reverse Forward Curriculum Learning for Extreme Sample and Demonstration Efficiency in Reinforcement Learning

arXiv 2405.03379 ↗First posted 6 May 2024cs.LG cs.AI cs.ROcited by 12

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning (RL) presents a promising framework to learn policies through environment interaction, but often requires an infeasible amount of interaction data to solve complex tasks from sparse rewards. One direction includes augmenting RL with offline data demonstrating desired tasks, but past work often require a lot of high-quality demonstration data that is difficult to obtain, especially for domains such as robotics. Our approach consists of a reverse curriculum followed by a forward curriculum. Unique to our approach compared to past work is the ability to efficiently leverage more than one demonstration via a per-demonstration reverse curriculum generated via state resets. The result of our reverse curriculum is an initial policy that performs well on a narrow initial state distribution and helps overcome difficult exploration problems. A forward curriculum is then used to accelerate the training of the initial policy to perform well on the full initial state distribution of the task and improve demonstration and sample efficiency. We show how the combination of a reverse curriculum and forward curriculum in our method, RFCL, enables significant improvements in demonstration and sample efficiency compared against various state-of-the-art learning-from-demonstration baselines, even solving previously unsolvable tasks that require high precision and control.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 32 of its 36 references that are in the map. First posted on arXiv 6 May 2024.

Builds on

32 of 36 references.

References · 32

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2016Concrete Problems in AI Safetycited by 1,582
  4. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  5. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  6. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  7. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  8. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  9. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  10. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 375
  11. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 350
  12. 2023Efficient Online Reinforcement Learning with Offline Datacited by 332
20 more
  1. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
  2. 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 313
  3. 2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 278
  4. 2023Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuningcited by 230
  5. 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 207
  6. 2022Online Decision Transformercited by 194
  7. 2020Prioritized Level Replaycited by 190
  8. 2017Reverse Curriculum Generation for Reinforcement Learningcited by 185
  9. 2022Masked World Models for Visual Controlcited by 169
  10. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skillscited by 124
  11. 2019End-to-End Robotic Reinforcement Learning without Reward Engineeringcited by 115
  12. 2022Jump-Start Reinforcement Learningcited by 102
  13. 2018Learning Montezuma's Revenge from a Single Demonstrationcited by 76
  14. 2017Data-efficient Deep Reinforcement Learning for Dexterous Manipulationcited by 61
  15. 2022MoDem: Accelerating Visual Model-Based Reinforcement Learning with Demonstrationscited by 59
  16. 2019Towards Practical Multi-Object Manipulation using Relational Reinforcement Learningcited by 54
  17. 2023Sequential Dexterity: Chaining Dexterous Policies for Long-Horizon Manipulationcited by 43
  18. 2016Playing Atari Games with Deep Reinforcement Learning and Human Checkpoint Replaycited by 22
  19. 2018BaRC: Backward Reachability Curriculum for Robotic Reinforcement Learningcited by 21
  20. 2018Backplay: "Man muss immer umkehren"cited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 5; 2026: 5.

12 papers.

  1. 2025Improving RL Exploration for LLM Reasoning through Retrospective Replaycited by 19
  2. 2024Continuous Control with Coarse-to-fine Reinforcement Learningcited by 15
  3. 2025RL for Reasoning by Adaptively Revealing Rationalescited by 15
  4. 2024DemoStart: Demonstration-led auto-curriculum applied to sim-to-real with multi-fingered robotscited by 7
  5. 2026RL-AWB: Deep Reinforcement Learning for Auto White Balance Correction in Low-Light Night-time Scenescited by 2
  6. 2025AURA: Autonomous Upskilling with Retrieval-Augmented Agentscited by 1
  7. 2025Enhanced Penalty-based Bidirectional Reinforcement Learning Algorithmscited by 0
  8. 2025Orchestrate, Generate, Reflect: A VLM-Based Multi-Agent Collaboration Framework for Automated Driving Policy Learningcited by 0
  9. 2026Level Up: Defining and Exploiting Transitional Problems for Curriculum Learningcited by 0
  10. 2026Grasp-Then-Plan with Failure Attribution: A Closed Two-Stage Framework for Precise and Generalizable Robotic Manipulationcited by 0
  11. 2026SCOUT: Per-Context Reset Curricula for Sparse-Reward Reinforcement Learningcited by 0
  12. 2026A Balanced Data Diet: Addressing the Exploration Bottleneck in Mega-Scale RL for Robot Controlcited by 0