# Revisiting Reward Design and Evaluation for Robust Humanoid Standing and Walking

arXiv: https://arxiv.org/abs/2404.19173 · HTML: https://mlsanity.com/paper/2404.19173/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Apr 2024.

## Built on by (most cited first, 21 of 21)

- [HumanPlus: Humanoid Shadowing and Imitation from Humans](https://mlsanity.com/paper/2406.10454/) (2024)
- [BeamDojo: Learning Agile Humanoid Locomotion on Sparse Footholds](https://mlsanity.com/paper/2502.10363/) (2025)
- [Learning Multi-Modal Whole-Body Control for Real-World Humanoid Robots](https://mlsanity.com/paper/2408.07295/) (2024)
- [DreamControl: Human-Inspired Whole-Body Humanoid Control for Scene Interaction via Guided Diffusion](https://mlsanity.com/paper/2509.14353/) (2025)
- [Learning Humanoid Locomotion with World Model Reconstruction](https://mlsanity.com/paper/2502.16230/) (2025)
- [FRASA: An End-to-End Reinforcement Learning Agent for Fall Recovery and Stand Up of Humanoid Robots](https://mlsanity.com/paper/2410.08655/) (2024)
- [SkillBlender: Towards Versatile Humanoid Whole-Body Loco-Manipulation via Skill Blending](https://mlsanity.com/paper/2506.09366/) (2025)
- [Embrace Collisions: Humanoid Shadowing for Deployable Contact-Agnostics Motions](https://mlsanity.com/paper/2502.01465/) (2025)
- [Natural Humanoid Robot Locomotion with Generative Motion Prior](https://mlsanity.com/paper/2503.09015/) (2025)
- [Non-conflicting Energy Minimization in Reinforcement Learning based Robot Control](https://mlsanity.com/paper/2509.01765/) (2025)
- [Learning Decentralized Multi-Biped Control for Payload Transport](https://mlsanity.com/paper/2406.17279/) (2024)
- [Multi-Embodiment Robotic Retargeting via Guided Diffusion Model](https://mlsanity.com/paper/2505.20857/) (2025)
- [No More Blind Spots: Learning Vision-Based Omnidirectional Bipedal Locomotion for Challenging Terrain](https://mlsanity.com/paper/2508.11929/) (2025)
- [Embedding Classical Balance Control Principles in Reinforcement Learning for Humanoid Recovery](https://mlsanity.com/paper/2603.08619/) (2026)
- [Robust RL Control for Bipedal Locomotion with Closed Kinematic Chains](https://mlsanity.com/paper/2507.10164/) (2025)
- [No More Marching: Learning Humanoid Locomotion for Short-Range SE(2) Targets](https://mlsanity.com/paper/2508.14098/) (2025)
- [E-SDS: Environment-aware See it, Do it, Sorted - Automated Environment-Aware Reinforcement Learning for Humanoid Locomotion](https://mlsanity.com/paper/2512.16446/) (2025)
- [A Hybrid Autoencoder for Robust Heightmap Generation from Fused Lidar and Depth Data for Humanoid Robot Locomotion](https://mlsanity.com/paper/2602.05855/) (2026)
- [Efficient Soft Actor-Critic with LLM-Based Action-Level Guidance for Continuous Control](https://mlsanity.com/paper/2603.17468/) (2026)
- [Adaptive Smooth Tchebycheff Attention for Multi-Objective Policy Optimization](https://mlsanity.com/paper/2605.12771/) (2026)
- [GaitSpan: Growing Humanoid Locomotion from Walking to Running](https://mlsanity.com/paper/2607.12114/) (2026)

## Abstract (arXiv)

A necessary capability for humanoid robots is the ability to stand and walk while rejecting natural disturbances. Recent progress has been made using sim-to-real reinforcement learning (RL) to train such locomotion controllers, with approaches differing mainly in their reward functions. However, prior works lack a clear method to systematically test new reward functions and compare controller performance through repeatable experiments. This limits our understanding of the trade-offs between approaches and hinders progress. To address this, we propose a low-cost, quantitative benchmarking method to evaluate and compare the real-world performance of standing and walking (SaW) controllers on metrics like command following, disturbance recovery, and energy efficiency. We also revisit reward function design and construct a minimally constraining reward function to train SaW controllers. We experimentally verify that our benchmarking framework can identify areas for improvement, which can be systematically addressed to enhance the policies. We also compare our new controller to state-of-the-art controllers on the Digit humanoid robot. The results provide clear quantitative trade-offs among the controllers and suggest directions for future improvements to the reward functions and expansion of the benchmarks.