mlsanityData to 14 Oct 2026

Reinforcement Learning via Auxiliary Task Distillation

arXiv 2406.17168 ↗First posted 24 Jun 2024cs.LG cs.AI cs.ROcited by 2

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

We present Reinforcement Learning via Auxiliary Task Distillation (AuxDistill), a new method that enables reinforcement learning (RL) to perform long-horizon robot control problems by distilling behaviors from auxiliary RL tasks. AuxDistill achieves this by concurrently carrying out multi-task RL with auxiliary tasks, which are easier to learn and relevant to the main task. A weighted distillation loss transfers behaviors from these auxiliary tasks to solve the main task. We demonstrate that AuxDistill can learn a pixels-to-actions policy for a challenging multi-stage embodied object rearrangement task from the environment reward without demonstrations, a learning curriculum, or pre-trained skills. AuxDistill achieves 2.3 × higher success than the previous state-of-the-art baseline in the Habitat Object Rearrangement benchmark and outperforms methods that use pre-trained skills and expert demonstrations.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 44 of its 51 references that are in the map. First posted on arXiv 24 Jun 2024.

Builds on

44 of 51 references.

References · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  6. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  7. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  8. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  9. 2019Solving Rubik's Cube with a Robot Handcited by 734
  10. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
  11. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 511
  12. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
32 more
  1. 2016The Option-Critic Architecturecited by 448
  2. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
  3. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 362
  4. 2019Emergent Tool Use From Multi-Agent Autocurriculacited by 313
  5. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
  6. 2020Rearrangement: A Challenge for Embodied AIcited by 155
  7. 2022Legged Locomotion in Challenging Terrains using Egocentric Visioncited by 151
  8. 2018Multi-task Deep Reinforcement Learning with PopArtcited by 141
  9. 2022Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotioncited by 121
  10. 2022In-Hand Object Rotation via Rapid Motor Adaptationcited by 118
  11. 2022Deep Hierarchical Planning from Pixelscited by 101
  12. 2023Generative Skill Chaining: Long-Horizon Skill Planning with Diffusion Modelscited by 91
  13. 2019Hindsight Credit Assignmentcited by 79
  14. 2021Auxiliary Tasks and Exploration Enable ObjectNavcited by 74
  15. 2019Shaping Belief States with Generative Environment Models for RLcited by 66
  16. 2021Accelerating Robotic Reinforcement Learning via Parameterized Action Primitivescited by 65
  17. 2023Gen2Sim: Scaling up Robot Learning in Simulation with Generative Modelscited by 64
  18. 2023When Do Transformers Shine in RL? Decoupling Memory from Credit Assignmentcited by 63
  19. 2020Auxiliary Tasks Speed Up Learning PointGoal Navigationcited by 54
  20. 2020ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulationcited by 49
  21. 2023Sequential Dexterity: Chaining Dexterous Policies for Long-Horizon Manipulationcited by 49
  22. 2021Adversarial Skill Chaining for Long-Horizon Robot Manipulation via Terminal State Regularizationcited by 46
  23. 2022Multi-skill Mobile Manipulation for Object Rearrangementcited by 46
  24. 2019DAC: The Double Actor-Critic Architecture for Learning Optionscited by 43
  25. 2022Improving Policy Optimization with Generalist-Specialist Learningcited by 20
  26. 2022CLUTR: Curriculum Learning via Unsupervised Task Representation Learningcited by 20
  27. 2017End-to-End Optimization of Task-Oriented Dialogue Model with Deep Reinforcement Learningcited by 19
  28. 2023Skill Transformer: A Monolithic Policy for Mobile Manipulationcited by 16
  29. 2022Structure-Encoding Auxiliary Tasks for Improved Visual Representation in Vision-and-Language Navigationcited by 9
  30. 2020Beyond Tabula-Rasa: a Modular Reinforcement Learning Approach for Physically Embedded 3D Sokobancited by 7
  31. 2022SkillS: Adaptive Skill Sequencing for Efficient Temporally-Extended Explorationcited by 6
  32. 2023Galactic: Scaling End-to-End Reinforcement Learning for Rearrangement at 100k Steps-Per-Secondcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Enhancing Lightweight Vision Language Models through Group Competitive Learning for Socially Compliant Navigationcited by 0
  2. 2026KD-MARL: Resource-Aware Knowledge Distillation in Multi-Agent Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Decoupling Task and Behavior: A Two-Stage Reward Curriculum in Reinforcement Learning for Robotics
  2. 2025Reward Training Wheels: Adaptive Auxiliary Rewards for Robotics Reinforcement Learning
  3. 2025Generating Auxiliary Tasks with Reinforcement Learning
  4. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning
  5. 2022Latent Plans for Task-Agnostic Offline Reinforcement Learning
  6. 2026DriftOPD: Sequence-Level Reverse-KL Distillation for One-Step VLA Policies
  7. 2026From Pretraining to Proficiency: Real-World Subtask RL for Long-Horizon Manipulation with Minimal Human Intervention
  8. 2026CIDER: Continual Interactive Distillation for Embodied Reinforcement Learning