mlsanityData to 13 Oct 2026

Making Efficient Use of Demonstrations to Solve Hard Exploration Problems

arXiv 1909.01387 ↗First posted 3 Sep 2019cs.LG cs.AIcited by 45

Not on the map. Why?

Abstract (arXiv)

This paper introduces R2D3, an agent that makes efficient use of demonstrations to solve hard exploration problems in partially observable environments with highly variable initial conditions. We also introduce a suite of eight tasks that combine these three properties, and show that R2D3 can solve several of the tasks where other state of the art methods (both with and without demonstrations) fail to see even a single successful trajectory after tens of billions of steps of exploration.

Key facts. Cited by 45 papers in the map (data through 13 Oct 2026). First posted on arXiv 3 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2019: 1; 2020: 3; 2021: 12; 2022: 8; 2023: 8; 2024: 4; 2025: 8; 2026: 1.

45 papers.

Sort:

By area:

  1. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 884
  2. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 158
  3. 2020Hyperparameter Selection for Offline Reinforcement Learningcited by 114
  4. 2020Offline Learning from Demonstrations and Unlabeled Experiencecited by 72
  5. 2022BYOL-Explore: Exploration by Bootstrapped Predictioncited by 71
  6. 2021What Matters for Adversarial Imitation Learning?cited by 70
  7. 2024Scaling Instructable Agents Across Many Simulated Worldscited by 69
  8. 2021Hit and Lead Discovery with Explorative RL and Fragment-based Molecule Generationcited by 68
  9. 2022Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progresscited by 64
  10. 2020RL Unplugged: A Suite of Benchmarks for Offline Reinforcement Learningcited by 57
  11. 2022Reinforcement Learning with Sparse Rewards using Guidance from Offline Demonstrationcited by 54
  12. 2021RLDS: an Ecosystem to Generate, Share and Use Datasets in Reinforcement Learningcited by 31
33 more
  1. 2025SIMA 2: A Generalist Embodied Agent for Virtual Worldscited by 28
  2. 2021PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learningcited by 23
  3. 2021Prioritized Experience-based Reinforcement Learning with Human Guidance for Autonomous Drivingcited by 21
  4. 2021Imitation by Predicting Observationscited by 13
  5. 2025Reinforcement Learning via Implicit Imitation Guidancecited by 12
  6. 2024VELO: A Vector Database-Assisted Cloud-Edge Collaborative LLM QoS Optimization Frameworkcited by 10
  7. 2023Unlocking the Power of Representations in Long-term Novelty-based Explorationcited by 8
  8. 2019ZPD Teaching Strategies for Deep Reinforcement Learning from Demonstrationscited by 5
  9. 2021Graph augmented Deep Reinforcement Learning in the GameRLand3D environmentcited by 5
  10. 2022Safe Decision-making for Lane-change of Autonomous Vehicles via Human Demonstration-aided Reinforcement Learningcited by 5
  11. 2022Cell-Free Latent Go-Explorecited by 5
  12. 2023Visual Hindsight Self-Imitation Learning for Interactive Navigationcited by 5
  13. 2022Self-Imitation Learning from Demonstrationscited by 4
  14. 2025Reinforcement Learning with Physics-Informed Symbolic Program Priors for Zero-Shot Wireless Indoor Navigationcited by 4
  15. 2021Did I do that? Blame as a means to identify controlled effects in reinforcement learningcited by 2
  16. 2023Selective Reincarnation: Offline-to-Online Multi-Agent Reinforcement Learningcited by 2
  17. 2023Representations and Exploration for Deep Reinforcement Learning using Singular Value Decompositioncited by 2
  18. 2023Adaptive trajectory-constrained exploration strategy for deep reinforcement learningcited by 2
  19. 2024WayEx: Waypoint Exploration using a Single Demonstrationcited by 2
  20. 2025LLM-Enhanced Multi-Agent Reinforcement Learning with Expert Workflow for Real-Time P2P Energy Tradingcited by 2
  21. 2021Lyapunov-based uncertainty-aware safe reinforcement learningcited by 1
  22. 2023Boosting Reinforcement Learning and Planning with Demonstrations: A Surveycited by 1
  23. 2025Mixture of Autoencoder Experts Guidance using Unlabeled and Incomplete Data for Exploration in Reinforcement Learningcited by 1
  24. 2021Reinforced Imitation Learning by Free Energy Principlecited by 0
  25. 2021DCUR: Data Curriculum for Teaching via Samples with Reinforcement Learningcited by 0
  26. 2022Extended Intelligencecited by 0
  27. 2023Hierarchical Reinforcement Learning in Complex 3D Environmentscited by 0
  28. 2023Policy Optimization with Smooth Guidance Learned from State-Only Demonstrationscited by 0
  29. 2024Single-Reset Divide & Conquer Imitation Learningcited by 0
  30. 2025Oh-A-DINO: Understanding and Enhancing Attribute-Level Information in Self-Supervised Object-Centric Representationscited by 0
  31. 2025Dynamic Action Interpolation: A Universal Approach for Accelerating Reinforcement Learning with Expert Guidancecited by 0
  32. 2025CAWR: Corruption-Averse Advantage-Weighted Regression for Robust Policy Optimizationcited by 0
  33. 2026Learning from the Test: Self-Referential Differential Testing for Deep RL Agentscited by 0