mlsanityData to 7 Oct 2026

Learning by Playing - Solving Sparse Reward Tasks from Scratch

arXiv 1802.10567 ↗First posted 28 Feb 2018cs.LG cs.RO stat.MLcited by 164

Not on the map. Why?

Abstract (arXiv)

We propose Scheduled Auxiliary Control (SAC-X), a new learning paradigm in the context of Reinforcement Learning (RL). SAC-X enables learning of complex behaviors - from scratch - in the presence of multiple sparse reward signals. To this end, the agent is equipped with a set of general auxiliary tasks, that it attempts to learn simultaneously via off-policy RL. The key idea behind our method is that active (learned) scheduling and execution of auxiliary policies allows the agent to efficiently explore its environment - enabling it to excel at sparse reward RL. Our experiments in several challenging robotic manipulation settings demonstrate the power of our approach.

Key facts. Cited by 164 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 1; 2021: 47; 2022: 31; 2023: 24; 2024: 27; 2025: 17; 2026: 16.

164 papers.

Sort:

By area:

  1. 2023Mastering Diverse Domains through World Modelscited by 1,193
  2. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 257
  3. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  4. 2024ReFT: Reasoning with Reinforced Fine-Tuningcited by 236
  5. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  6. 2020Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learningcited by 213
  7. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  8. 2019Goal-conditioned Imitation Learningcited by 159
  9. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 158
  10. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 153
  11. 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 111
  12. 2021The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixelscited by 97
38 more
  1. 2025Reinforcement Learning with Action Chunkingcited by 97
  2. 2022Don't Change the Algorithm, Change the Data: Exploratory Data for Offline Reinforcement Learningcited by 93
  3. 2021Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Interventioncited by 90
  4. 2021From Motor Control to Team Play in Simulated Humanoid Footballcited by 87
  5. 2022Latent Plans for Task-Agnostic Offline Reinforcement Learningcited by 81
  6. 2023Video Prediction Models as Rewards for Reinforcement Learningcited by 76
  7. 2021Auxiliary Tasks and Exploration Enable ObjectNavcited by 73
  8. 2023For SALE: State-Action Representation Learning for Deep Reinforcement Learningcited by 73
  9. 2021Asymmetric self-play for automatic goal discovery in robotic manipulationcited by 71
  10. 2021Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapescited by 70
  11. 2022CACTI: A Framework for Scalable Multi-Task Multi-Scene Visual Imitation Learningcited by 70
  12. 2021Hierarchical Reinforcement Learning By Discovering Intrinsic Optionscited by 57
  13. 2021On The Effect of Auxiliary Tasks on Representation Dynamicscited by 56
  14. 2021Adversarial Skill Chaining for Long-Horizon Robot Manipulation via Terminal State Regularizationcited by 44
  15. 2021Hierarchical Skills for Efficient Explorationcited by 37
  16. 2023Contrastive Difference Predictive Codingcited by 35
  17. 2021Self-Supervised Online Reward Shaping in Sparse-Reward Environmentscited by 34
  18. 2021Lifelong Robotic Reinforcement Learning by Retaining Experiencescited by 34
  19. 2021ScheduleNet: Learn to solve multi-agent scheduling problems with reinforcement learningcited by 31
  20. 2024Offline Actor-Critic Reinforcement Learning Scales to Large Modelscited by 30
  21. 2022Curious Exploration via Structured World Models Yields Zero-Shot Object Manipulationcited by 29
  22. 2025Towards Better Alignment: Training Diffusion Models with Reinforcement Learning Against Sparse Rewardscited by 29
  23. 2021Rank the Episodes: A Simple Approach for Exploration in Procedurally-Generated Environmentscited by 28
  24. 2022Human-level Atari 200x fastercited by 28
  25. 2021Discovery of Options via Meta-Learned Subgoalscited by 27
  26. 2021Learning Transferable Motor Skills with Hierarchical Latent Mixture Policiescited by 26
  27. 2021Return-based Scaling: Yet Another Normalisation Trick for Deep RLcited by 22
  28. 2024Enhancing Multi-Step Reasoning Abilities of Language Models through Direct Q-Function Optimizationcited by 22
  29. 2021A Probabilistic Interpretation of Self-Paced Learning with Applications to Reinforcement Learningcited by 21
  30. 2021Learning Multi-Stage Tasks with One Demonstration via Self-Replaycited by 21
  31. 2022Intelligent problem-solving as integrated hierarchical reinforcement learningcited by 21
  32. 2021An Information-Theoretic Perspective on Credit Assignment in Reinforcement Learningcited by 20
  33. 2022Deep Reinforcement Learning Guided Improvement Heuristic for Job Shop Schedulingcited by 20
  34. 2021Solving Compositional Reinforcement Learning Problems via Task Reductioncited by 19
  35. 2023Proximal Curriculum for Reinforcement Learning Agentscited by 19
  36. 2021Collect & Infer -- a fresh look at data-efficient Reinforcement Learningcited by 17
  37. 2022Critic Sequential Monte Carlocited by 16
  38. 2023Blending Imitation and Reinforcement Learning for Robust Policy Improvementcited by 16

50 most cited of 164 All 164 (JSON)