mlsanityData to 7 Oct 2026

Concurrent Training of a Control Policy and a State Estimator for Dynamic and Robust Legged Locomotion

arXiv 2202.05481 ↗First posted 11 Feb 2022cs.RO cs.LG cs.SYcited by 61

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose a locomotion training framework where a control policy and a state estimator are trained concurrently. The framework consists of a policy network which outputs the desired joint positions and a state estimation network which outputs estimates of the robot's states such as the base linear velocity, foot height, and contact probability. We exploit a fast simulation environment to train the networks and the trained networks are transferred to the real robot. The trained policy and state estimator are capable of traversing diverse terrains such as a hill, slippery plate, and bumpy road. We also demonstrate that the learned policy can run at up to 3.75 m/s on normal flat ground and 3.54 m/s on a slippery plate with the coefficient of friction of 0.22.

Key facts. Cited by 61 papers in the map (data through 7 Oct 2026). Builds on 14 of its 23 references that are in the map. First posted on arXiv 11 Feb 2022.

Builds on

14 of 23 references.

References · 14

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  3. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 494
  4. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 430
  5. 2019Learning agile and dynamic motor skills for legged robotscited by 402
  6. 2020Learning Agile Robotic Locomotion Skills by Imitating Animalscited by 229
  7. 2021Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learningcited by 73
  8. 2020Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward Compositioncited by 71
  9. 2020Multi-expert learning of adaptive legged locomotioncited by 62
  10. 2019Highly Dynamic Quadruped Locomotion via Whole-Body Impulse Control and Model Predictive Controlcited by 43
  11. 2019Robust Recovery Controller for a Quadrupedal Robot using Deep Reinforcement Learningcited by 30
  12. 2020Learning to Locomote: Understanding How Environment Design Matters for Deep Reinforcement Learningcited by 27
2 more
  1. 2019Contact-Aided Invariant Extended Kalman Filtering for Robot State Estimationcited by 16
  2. 2020Representation-Free Model Predictive Control for Dynamic Motions in Quadrupedscited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 9; 2023: 18; 2024: 15; 2025: 13; 2026: 6.

61 papers.

Sort:

By area:

  1. 2022Rapid Locomotion via Reinforcement Learningcited by 130
  2. 2023Robot Parkour Learningcited by 115
  3. 2022Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behaviorcited by 113
  4. 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 111
  5. 2025ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skillscited by 104
  6. 2022Adversarial Motion Priors Make Good Substitutes for Complex Reward Functionscited by 68
  7. 2024Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Controlcited by 63
  8. 2022Advanced Skills by Learning Locomotion and Local Navigation End-to-Endcited by 39
  9. 2022GenLoco: Generalized Locomotion Controllers for Quadrupedal Robotscited by 36
  10. 2024One Policy to Run Them All: an End-to-end Learning Approach to Multi-Embodiment Locomotioncited by 34
  11. 2023DribbleBot: Dynamic Legged Manipulation in the Wildcited by 33
  12. 2023Not Only Rewards But Also Constraints: Applications on Legged Robot Locomotioncited by 33
38 more
  1. 2022Hierarchical Reinforcement Learning for Precise Soccer Shooting Skills using a Quadrupedal Robotcited by 32
  2. 2024Learning Robust Autonomous Navigation and Locomotion for Wheeled-Legged Robotscited by 32
  3. 2023Legs as Manipulator: Pushing Quadrupedal Agility Beyond Locomotioncited by 30
  4. 2023Robust and Versatile Bipedal Jumping Control through Reinforcement Learningcited by 27
  5. 2024Learning Force Control for Legged Manipulationcited by 19
  6. 2022Creating a Dynamic Quadrupedal Robotic Goalkeeper with Reinforcement Learningcited by 17
  7. 2025Learning coordinated badminton skills for legged manipulatorscited by 16
  8. 2022Learning and Deploying Robust Locomotion Policies with Minimal Dynamics Randomizationcited by 15
  9. 2023ManyQuadrupeds: Learning a Single Locomotion Policy for Diverse Quadruped Robotscited by 15
  10. 2024Learning Visual Quadrupedal Loco-Manipulation from Demonstrationscited by 15
  11. 2023Controlling the Solo12 Quadruped Robot with Deep Reinforcement Learningcited by 11
  12. 2025Bridging the Sim-to-Real Gap for Athletic Loco-Manipulationcited by 11
  13. 2024Stage-Wise Reward Shaping for Acrobatic Robots: A Constrained Multi-Objective Reinforcement Learning Approachcited by 10
  14. 2025Learning Humanoid Locomotion with World Model Reconstructioncited by 10
  15. 2024Helpful DoggyBot: Open-World Object Fetching using Legged Robots and Vision-Language Modelscited by 8
  16. 2022Learning Low-Frequency Motion Control for Robust and Dynamic Robot Locomotioncited by 7
  17. 2023Learning Robust, Agile, Natural Legged Locomotion Skills in the Wildcited by 7
  18. 2023Adapt On-the-Go: Behavior Modulation for Single-Life Robot Deploymentcited by 7
  19. 2023Learning to See Physical Properties with Active Sensing Motor Policiescited by 7
  20. 2025Gait in Eight: Efficient On-Robot Learning for Omnidirectional Quadruped Locomotioncited by 7
  21. 2023Puppeteer and Marionette: Learning Anticipatory Quadrupedal Locomotion Based on Interactions of a Central Pattern Generator and Supraspinal Drivecited by 5
  22. 2024Whole-body End-Effector Pose Trackingcited by 5
  23. 2025High-Performance Reinforcement Learning on Spot: Optimizing Simulation Parameters with Distributional Measurescited by 5
  24. 2026FlashSAC: Fast and Stable Off-Policy Reinforcement Learning for High-Dimensional Robot Controlcited by 5
  25. 2023Identifying Important Sensory Feedback for Learning Locomotion Skillscited by 4
  26. 2024QuadrupedGPT: Towards a Versatile Quadruped Agent in Open-ended Worldscited by 4
  27. 2025FACET: Force-Adaptive Control via Impedance Reference Tracking for Legged Robotscited by 4
  28. 2023Seeing-Eye Quadruped Navigation with Force Responsive Locomotion Controlcited by 3
  29. 2024Legged Robot State Estimation With Invariant Extended Kalman Filter Using Neural Measurement Networkcited by 3
  30. 2025PrivilegedDreamer: Explicit Imagination of Privileged Information for Rapid Adaptation of Learned Policiescited by 3
  31. 2024DexDribbler: Learning Dexterous Soccer Manipulation via Dynamic Supervisioncited by 2
  32. 2023Roll-Drop: accounting for observation noise with a single parametercited by 1
  33. 2024Learning Rapid Turning, Aerial Reorientation, and Balancing using Manipulator as a Tailcited by 1
  34. 2024QuadWBG: Generalizable Quadrupedal Whole-Body Graspingcited by 1
  35. 2025Unified Locomotion Transformer with Simultaneous Sim-to-Real Transfer for Quadrupedscited by 1
  36. 2026AdaptManip: Learning Adaptive Whole-Body Object Lifting and Delivery with Online Recurrent State Estimationcited by 1
  37. 2026Agile perceptive multi-skill locomotion for quadrupedal robots in the wildcited by 1
  38. 2023Combining model-predictive control and predictive reinforcement learning for stable quadrupedal robot locomotioncited by 0

50 most cited of 61 All 61 (JSON)