mlsanityData to 14 Oct 2026

Stage-Wise Reward Shaping for Acrobatic Robots: A Constrained Multi-Objective Reinforcement Learning Approach

arXiv 2409.15755 ↗First posted 24 Sep 2024cs.RO cs.AIcited by 13

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

As the complexity of tasks addressed through reinforcement learning (RL) increases, the definition of reward functions also has become highly complicated. We introduce an RL method aimed at simplifying the reward-shaping process through intuitive strategies. Initially, instead of a single reward function composed of various terms, we define multiple reward and cost functions within a constrained multi-objective RL (CMORL) framework. For tasks involving sequential complex movements, we segment the task into distinct stages and define multiple rewards and costs for each stage. Finally, we introduce a practical CMORL algorithm that maximizes objectives based on these rewards while satisfying constraints defined by the costs. The proposed method has been successfully demonstrated across a variety of acrobatic tasks in both simulation and real-world environments. Additionally, it has been shown to successfully perform tasks compared to existing RL and constrained RL algorithms. Our code is available at https://github.com/rllab-snu/Stage-Wise-CMORL.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 24 of its 34 references that are in the map. First posted on arXiv 24 Sep 2024.

Builds on

24 of 34 references.

References · 24

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Trust Region Policy Optimizationcited by 2,580
  3. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  4. 2017Constrained Policy Optimizationcited by 983
  5. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  6. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
  7. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 316
  8. 2021A Practical Guide to Multi-Objective Reinforcement Learning and Planningcited by 256
  9. 2019A Generalized Algorithm for Multi-Objective Reinforcement Learning and Policy Adaptationcited by 177
  10. 2023Extreme Parkour with Legged Robotscited by 164
  11. 2018DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skillscited by 163
  12. 2022Legged Locomotion in Challenging Terrains using Egocentric Visioncited by 151
12 more
  1. 2022Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behaviorcited by 138
  2. 2021Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learningcited by 92
  3. 2022Concurrent Training of a Control Policy and a State Estimator for Dynamic and Robust Legged Locomotioncited by 88
  4. 2021Legged Robots that Keep on Learning: Fine-Tuning Locomotion Policies in the Real Worldcited by 71
  5. 2022Penalized Proximal Policy Optimization for Safe Reinforcement Learningcited by 68
  6. 2022PD-MORL: Preference-Driven Multi-Objective Reinforcement Learning Algorithmcited by 58
  7. 2020A Distributional View on Multi-Objective Policy Optimizationcited by 52
  8. 2023Not Only Rewards But Also Constraints: Applications on Legged Robot Locomotioncited by 47
  9. 2023Barkour: Benchmarking Animal-level Agility with Quadruped Robotscited by 41
  10. 2023Robust and Versatile Bipedal Jumping Control through Reinforcement Learningcited by 36
  11. 2023Learning Agile Locomotion on Risky Terrainscited by 21
  12. 2023Evaluation of Constrained Reinforcement Learning Algorithms for Legged Locomotioncited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 5.

13 papers.

Sort:

By area:

  1. 2025Learning Humanoid Standing-up Control across Diverse Posturescited by 46
  2. 2025STARE-VLA: Progressive Stage-Aware Reinforcement for Fine-Tuning Vision-Language-Action Modelscited by 24
  3. 2025Gait in Eight: Efficient On-Robot Learning for Omnidirectional Quadruped Locomotioncited by 8
  4. 2025MARG: MAstering Risky Gap Terrains for Legged Robots with Elevation Mappingcited by 4
  5. 2025LAPP: Large Language Model Feedback for Preference-Driven Reinforcement Learningcited by 2
  6. 2025Bridge the Gap: Enhancing Quadruped Locomotion with Vertical Ground Perturbationscited by 1
  7. 2025Parental Guidance: Efficient Lifelong Learning through Evolutionary Distillationcited by 0
  8. 2025Learning Impact-Rich Rotational Maneuvers via Centroidal Velocity Rewards and Sim-to-Real Techniques: A One-Leg Hopper Flip Case Studycited by 0
  9. 2026Co-jump: Cooperative Jumping with Quadrupedal Robots via Multi-Agent Reinforcement Learningcited by 0
  10. 2026Reward-Zero: Language Embedding Driven Implicit Reward Mechanisms for Reinforcement Learningcited by 0
  11. 2026Learning Task-Invariant Properties via Dreamer: Enabling Efficient Policy Transfer for Quadruped Robotscited by 0
  12. 2026Robust Fall Recovery for Armless Bipedal-Wheeled Robots Via Force-Guided Learningcited by 0
1 more
  1. 2026Where Success Breaks: Failure-Boundary Learning for Robust Vision-Language-Action Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Constraints as Rewards: Reinforcement Learning for Robots without Reward Functions
  2. 2025Automated Hybrid Reward Scheduling via Large Language Models for Robotic Skill Learning
  3. 2024Curriculum Reinforcement Learning for Complex Reward Functions
  4. 2026Sufficiency of Zeroth-Order Reward Shaping for Policy Gradient in Stabilization Control
  5. 2025Automatic Reward Shaping from Multi-Objective Human Heuristics
  6. 2026Decoupling Task and Behavior: A Two-Stage Reward Curriculum in Reinforcement Learning for Robotics
  7. 2024To the Max: Reinventing Reward in Reinforcement Learning
  8. 2021Direct Behavior Specification via Constrained Reinforcement Learning