mlsanityData to 14 Oct 2026

Reward Training Wheels: Adaptive Auxiliary Rewards for Robotics Reinforcement Learning

arXiv 2503.15724 ↗First posted 19 Mar 2025cs.RO cs.AIcited by 4

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

Robotics Reinforcement Learning (RL) often relies on carefully engineered auxiliary rewards to supplement sparse primary learning objectives to compensate for the lack of large-scale, real-world, trial-and-error data. While these auxiliary rewards accelerate learning, they require significant engineering effort, may introduce human biases, and cannot adapt to the robot's evolving capabilities during training. In this paper, we introduce Reward Training Wheels (RTW), a teacher-student framework that automates auxiliary reward adaptation for robotics RL. To be specific, the RTW teacher dynamically adjusts auxiliary reward weights based on the student's evolving capabilities to determine which auxiliary reward aspects require more or less emphasis to improve the primary objective. We demonstrate RTW on two challenging robot tasks: navigation in highly constrained spaces and off-road vehicle mobility on vertically challenging terrain. In simulation, RTW outperforms expert-designed rewards by 2.35% in navigation success rate and improves off-road mobility performance by 122.62%, while achieving 35% and 3X faster training efficiency, respectively. Physical robot experiments further validate RTW's effectiveness, achieving a perfect success rate (5/5 trials vs. 2/5 for expert-designed rewards) and improving vehicle stability with up to 47.4% reduction in orientation angles.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 16 of its 33 references that are in the map. First posted on arXiv 19 Mar 2025.

Builds on

16 of 33 references.

References · 16

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2019Learning agile and dynamic motor skills for legged robotscited by 504
  3. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
  4. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 259
  5. 2017Inverse Reward Designcited by 249
  6. 2018Learning by Playing - Solving Sparse Reward Tasks from Scratchcited by 167
  7. 2022Unpacking Reward Shaping: Understanding the Benefits of Reward Engineering on Sample Complexitycited by 62
  8. 2018BaRC: Backward Reachability Curriculum for Robotic Reinforcement Learningcited by 23
  9. 2020Benchmarking Metric Ground Navigationcited by 20
  10. 2022Benchmarking Reinforcement Learning Techniques for Autonomous Navigationcited by 19
  11. 2023Toward Wheeled Mobility on Vertically Challenging Terrain: Platforms, Datasets, and Algorithmscited by 19
  12. 2024Reinforcement Learning for Wheeled Mobility on Vertically Challenging Terraincited by 7
4 more
  1. 2024Grounded Curriculum Learningcited by 5
  2. 2024Curriculum Reinforcement Learning for Complex Reward Functionscited by 5
  3. 2024Informativeness of Reward Functions in Reinforcement Learningcited by 3
  4. 2024VertiSelector: Automatic Curriculum Learning for Wheeled Mobility on Vertically Challenging Terraincited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025ELMUR: External Layer Memory with Update/Rewrite for Long-Horizon RL Problemscited by 6
  2. 2025DREAM: Domain-aware Reasoning for Efficient Autonomous Underwater Monitoringcited by 2
  3. 2026APPLV: Adaptive Planner Parameter Learning from Vision-Language-Action Modelcited by 0
  4. 2026A Unified Framework for Dynamic Reward Shaping in Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Reinforcement Learning via Auxiliary Task Distillation
  2. 2026Decoupling Task and Behavior: A Two-Stage Reward Curriculum in Reinforcement Learning for Robotics
  3. 2026From Demonstrations to Rewards: Test-Time Prompt Optimization for VLM Reward Models
  4. 2024To the Max: Reinventing Reward in Reinforcement Learning
  5. 2025Residual Reward Models: Leveraging Prior Knowledge for Efficient Preference-based Reinforcement Learning in Robotics
  6. 2024Auxiliary Reward Generation with Transition Distance Representation Learning
  7. 2025Automated Hybrid Reward Scheduling via Large Language Models for Robotic Skill Learning
  8. 2025Constraints as Rewards: Reinforcement Learning for Robots without Reward Functions