mlsanityData to 14 Oct 2026

Learning Locomotion Skills Using DeepRL: Does the Choice of Action Space Matter?

arXiv 1611.01055 ↗First posted 3 Nov 2016cs.LG cs.GR cs.ROcited by 48

Not on the map. Why?

Abstract (arXiv)

The use of deep reinforcement learning allows for high-dimensional state descriptors, but little is known about how the choice of action representation impacts the learning difficulty and the resulting performance. We compare the impact of four different action parameterizations (torques, muscle-activations, target joint angles, and target joint-angle velocities) in terms of learning time, policy robustness, motion quality, and policy query rates. Our results are evaluated on a gait-cycle imitation task for multiple planar articulated figures and multiple gaits. We demonstrate that the local feedback provided by higher-level action parameterizations can significantly impact the learning, robustness, and quality of the resulting policies.

Key facts. Cited by 48 papers in the map (data through 14 Oct 2026). First posted on arXiv 3 Nov 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 9; 2022: 14; 2023: 10; 2024: 8; 2025: 5; 2026: 2.

48 papers.

Sort:

By area:

  1. 2022PhysDiff: Physics-Guided Human Motion Diffusion Modelcited by 368
  2. 2021SimPoE: Simulated Character Control for 3D Human Pose Estimationcited by 114
  3. 2021Fast and Efficient Locomotion via Learned Gait Transitionscited by 54
  4. 2022Factory: Fast Contact for Robotic Assemblycited by 54
  5. 2021Learning to Jump from Pixelscited by 45
  6. 2023Lifelike Agility and Play in Quadrupedal Robots using Reinforcement Learning and Generative Pre-trained Modelscited by 37
  7. 2022Learning Torque Control for Quadrupedal Locomotioncited by 29
  8. 2023On the Role of the Action Space in Robot Manipulation Learning and Sim-to-Real Transfercited by 26
  9. 2022PoseTriplet: Co-evolving 3D Human Pose Estimation, Imitation, and Hallucination under Self-supervisioncited by 25
  10. 2021Robust High-speed Running for Quadruped Robots via Deep Reinforcement Learningcited by 22
  11. 2024HUMOS: Human Motion Model Conditioned on Body Shapecited by 20
  12. 2021Model-free Reinforcement Learning for Robust Locomotion using Demonstrations from Trajectory Optimizationcited by 19
36 more
  1. 2022Learning and Deploying Robust Locomotion Policies with Minimal Dynamics Randomizationcited by 17
  2. 2023Controlling the Solo12 Quadruped Robot with Deep Reinforcement Learningcited by 16
  3. 2022A Survey on Reinforcement Learning Methods in Character Animationcited by 12
  4. 2022Learning Low-Frequency Motion Control for Robust and Dynamic Robot Locomotioncited by 12
  5. 2022Generative GaitNetcited by 11
  6. 2022Learning to Get Upcited by 11
  7. 2025Shape My Moves: Text-Driven Shape-Aware Synthesis of Human Motionscited by 11
  8. 2021OstrichRL: A Musculoskeletal Ostrich Simulation to Study Bio-mechanical Locomotioncited by 10
  9. 2021Learning Bipedal Robot Locomotion from Human Movementcited by 7
  10. 2023DecAP: Decaying Action Priors for Accelerated Imitation Learning of Torque-Based Legged Locomotion Policiescited by 7
  11. 2024Learning the Optimal Power Flow: Environment Design Matterscited by 7
  12. 2024Deep Reinforcement Learning for Bipedal Locomotion: A Brief Surveycited by 7
  13. 2024Residual-MPPI: Online Policy Customization for Continuous Controlcited by 7
  14. 2024Automatic Environment Shaping is the Next Frontier in RLcited by 7
  15. 2022Learning with Muscles: Benefits for Data-Efficiency and Robustness in Anthropomorphic Taskscited by 6
  16. 2022CLAS: Coordinating Multi-Robot Manipulation with Central Latent Action Spacescited by 6
  17. 2023Identifying Important Sensory Feedback for Learning Locomotion Skillscited by 6
  18. 2021Learning Robotic Manipulation Skills Using an Adaptive Force-Impedance Action Spacecited by 5
  19. 2025Optimizing Bipedal Locomotion for The 100m Dash With Comparison to Human Runningcited by 5
  20. 2025Guided Reinforcement Learning for Omnidirectional 3D Jumping in Quadruped Robotscited by 4
  21. 2023Roll-Drop: accounting for observation noise with a single parametercited by 3
  22. 2023A Real-World Quadrupedal Locomotion Benchmark for Offline Reinforcement Learningcited by 3
  23. 2023Graph Agent Network: Empowering Nodes with Inference Capabilities for Adversarial Resiliencecited by 2
  24. 2023Investigating the Impact of Action Representations in Policy Gradient Algorithmscited by 2
  25. 2022Detecting danger in gridworlds using Gromov's Link Conditioncited by 1
  26. 2022Co-design of Embodied Neural Intelligence via Constrained Evolutioncited by 1
  27. 2023Efficient Reinforcement Learning for Jumping Monopodscited by 1
  28. 2024Deep Dive into Model-free Reinforcement Learning for Biological and Robotic Systems: Theory and Practicecited by 1
  29. 2024Exciting Action: Investigating Efficient Exploration for Learning Musculoskeletal Humanoid Locomotioncited by 1
  30. 2025Sim-to-Real Transfer in Deep Reinforcement Learning for Bipedal Locomotioncited by 1
  31. 2026Benchmarking Action Spaces in Reinforcement Learning for Vision-based Robotic Manipulationcited by 1
  32. 2021Learning Control Policies for Imitating Human Gaitscited by 0
  33. 2022Custom Sine Waves Are Enough for Imitation Learning of Bipedal Gaits with Different Stylescited by 0
  34. 2024Latent Action Priors for Locomotion with Deep Reinforcement Learningcited by 0
  35. 2025A General Approach of Automated Environment Design for Learning the Optimal Power Flowcited by 0
  36. 2026Reflex-Informed Neuromuscular Reinforcement Learning for Muscle-Driven Locomotioncited by 0