mlsanityData to 14 Oct 2026

Adversarial Motion Priors Make Good Substitutes for Complex Reward Functions

arXiv 2203.15103 ↗First posted 28 Mar 2022cs.AI cs.ROcited by 89

Not on the map. Why?

Cited by 36 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Training a high-dimensional simulated agent with an under-specified reward function often leads the agent to learn physically infeasible strategies that are ineffective when deployed in the real world. To mitigate these unnatural behaviors, reinforcement learning practitioners often utilize complex reward functions that encourage physically plausible behaviors. However, a tedious labor-intensive tuning process is often required to create hand-designed rewards which might not easily generalize across platforms and tasks. We propose substituting complex reward functions with "style rewards" learned from a dataset of motion capture demonstrations. A learned style reward can be combined with an arbitrary task reward to train policies that perform tasks using naturalistic strategies. These natural strategies can also facilitate transfer to the real world. We build upon Adversarial Motion Priors -- an approach from the computer graphics domain that encodes a style reward from a dataset of reference motions -- to demonstrate that an adversarial approach to training policies can produce behaviors that transfer to a real quadrupedal robot without requiring complex reward functions. We also demonstrate that an effective style reward can be learned from a few seconds of motion capture data gathered from a German Shepherd and leads to energy-efficient locomotion strategies with natural gait transitions.

Key facts. Cited by 89 papers in the map (data through 14 Oct 2026). Builds on 31 of its 57 references that are in the map. First posted on arXiv 28 Mar 2022.

Builds on

31 of 57 references.

References · 31

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2016Generative Adversarial Imitation Learningcited by 1,597
  3. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  4. 2016Least Squares Generative Adversarial Networkscited by 1,001
  5. 2019Solving Rubik's Cube with a Robot Handcited by 734
  6. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  7. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
  8. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
  9. 2016f-GAN: Training Generative Neural Samplers using Variational Divergence Minimizationcited by 506
  10. 2022Learning robust perceptive locomotion for quadrupedal robots in the wildcited by 382
  11. 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 366
  12. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 319
19 more
  1. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 316
  2. 2018Reward learning from human preferences and demonstrations in Ataricited by 316
  3. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 289
  4. 2017Emergence of Locomotion Behaviours in Rich Environmentscited by 165
  5. 2019Imitation Learning as f-Divergence Minimizationcited by 123
  6. 2021Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robotscited by 116
  7. 2020Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward Compositioncited by 96
  8. 2018GPU-Accelerated Robotic Simulation for Distributed Reinforcement Learningcited by 90
  9. 2022Concurrent Training of a Control Policy and a State Estimator for Dynamic and Robust Legged Locomotioncited by 88
  10. 2017Robust Imitation of Diverse Behaviorscited by 86
  11. 2017Learning human behaviors from motion capture by adversarial imitationcited by 77
  12. 2018Learning Symmetric and Low-energy Locomotioncited by 61
  13. 2021Fast and Efficient Locomotion via Learned Gait Transitionscited by 54
  14. 2021Coupling Vision and Proprioception for Navigation of Legged Robotscited by 51
  15. 2019Policies Modulating Trajectory Generatorscited by 49
  16. 2019Robust Recovery Controller for a Quadrupedal Robot using Deep Reinforcement Learningcited by 32
  17. 20163D Simulation for Robot Arm Control with Deep Q-Learningcited by 21
  18. 2020Learning Agile Locomotion Skills with a Mentorcited by 8
  19. 2020A Factor-Graph Approach for Optimization Problems with Dynamics Constraintscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 6; 2023: 16; 2024: 25; 2025: 26; 2026: 16.

89 papers.

Sort:

By area:

  1. 2022DayDreamer: World Models for Physical Robot Learningcited by 400
  2. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 176
  3. 2024Open-TeleVision: Teleoperation with Immersive Active Visual Feedbackcited by 145
  4. 2024Expressive Whole-Body Control for Humanoid Robotscited by 139
  5. 2024Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Controlcited by 99
  6. 2024ExBody2: Advanced Expressive Humanoid Whole-Body Controlcited by 88
  7. 2023PhysHOI: Physics-Based Imitation of Dynamic Human-Object Interactioncited by 76
  8. 2024Visual Whole-Body Control for Legged Loco-Manipulationcited by 62
  9. 2024Mobile-TeleVision: Predictive Motion Priors for Humanoid Whole-Body Controlcited by 51
  10. 2025AMO: Adaptive Motion Optimization for Hyper-Dexterous Humanoid Whole-Body Controlcited by 51
  11. 2024DiffuseLoco: Real-Time Legged Locomotion Control with Diffusion from Offline Datasetscited by 49
  12. 2023Not Only Rewards But Also Constraints: Applications on Legged Robot Locomotioncited by 47
38 more
  1. 2022Learning Agile Skills via Adversarial Imitation of Rough Partial Demonstrationscited by 45
  2. 2023Language Reward Modulation for Pretraining Reinforcement Learningcited by 43
  3. 2023Hybrid Internal Model: Learning Agile Legged Locomotion with Simulated Robot Responsecited by 39
  4. 2023Robust and Versatile Bipedal Jumping Control through Reinforcement Learningcited by 36
  5. 2023Learning and Adapting Agile Locomotion Skills by Transferring Experiencecited by 33
  6. 2022Imitate and Repurpose: Learning Reusable Robot Movement Skills From Human and Animal Behaviorscited by 30
  7. 2022Fleet-DAgger: Interactive Robot Fleet Learning with Scalable Human Supervisioncited by 30
  8. 2023RoboPianist: Dexterous Piano Playing with Deep Reinforcement Learningcited by 30
  9. 2024Whole-body Humanoid Robot Locomotion with Human Referencecited by 29
  10. 2023RL + Model-based Control: Using On-demand Optimal Control to Learn Versatile Legged Locomotioncited by 26
  11. 2022Creating a Dynamic Quadrupedal Robotic Goalkeeper with Reinforcement Learningcited by 23
  12. 2024CaT: Constraints as Terminations for Legged Locomotion Reinforcement Learningcited by 23
  13. 2024Curriculum-Based Reinforcement Learning for Quadrupedal Jumping: A Reference-free Designcited by 18
  14. 2024RobotKeyframing: Learning Locomotion with High-Level Objectives via Mixture of Dense and Sparse Rewardscited by 18
  15. 2024World Model-based Perception for Visual Legged Locomotioncited by 18
  16. 2025Design and Control of a Bipedal Robotic Charactercited by 18
  17. 2025PhysHSI: Towards a Real-World Generalizable and Natural Humanoid-Scene Interaction Systemcited by 18
  18. 2024A Learning Framework for Diverse Legged Robot Locomotion Using Barrier-Based Style Rewardscited by 12
  19. 2025Multi-Stage Manipulation with Demonstration-Augmented Reward, Policy, and World Model Learningcited by 11
  20. 2023Bridging the Sim-to-Real Gap from the Information Bottleneck Perspectivecited by 10
  21. 2024Guided Reinforcement Learning for Robust Multi-Contact Loco-Manipulationcited by 10
  22. 2025MoRE: Mixture of Residual Experts for Humanoid Lifelike Gaits Learning on Complex Terrainscited by 10
  23. 2025SkillBlender: Towards Versatile Humanoid Whole-Body Loco-Manipulation via Skill Blendingcited by 9
  24. 2023Residual Physics Learning and System Identification for Sim-to-real Transfer of Policies on Buoyancy Assisted Legged Robotscited by 8
  25. 2023Learning Robust, Agile, Natural Legged Locomotion Skills in the Wildcited by 8
  26. 2024PA-LOCO: Learning Perturbation-Adaptive Locomotion for Quadruped Robotscited by 8
  27. 2024Preference Aligned Diffusion Planner for Quadrupedal Locomotion Controlcited by 7
  28. 2025Embrace Collisions: Humanoid Shadowing for Deployable Contact-Agnostics Motionscited by 7
  29. 2025Gait-Conditioned Reinforcement Learning with Multi-Phase Curriculum for Humanoid Locomotioncited by 6
  30. 2025Motion Priors Reimagined: Adapting Flat-Terrain Skills for Complex Quadruped Mobilitycited by 5
  31. 2025Non-conflicting Energy Minimization in Reinforcement Learning based Robot Controlcited by 5
  32. 2023Decentralized Motor Skill Learning for Complex Robotic Systemscited by 4
  33. 2024HiLMa-Res: A General Hierarchical Framework via Residual RL for Combining Quadrupedal Locomotion and Manipulationcited by 4
  34. 2025AcL: Action Learner for Fault-Tolerant Quadruped Locomotion Controlcited by 3
  35. 2025Integrating Diffusion-based Multi-task Learning with Online Reinforcement Learning for Robust Quadruped Robot Controlcited by 3
  36. 2025Learning to Generate Pointing Gestures in Situated Embodied Conversational Agentscited by 3
  37. 2025Latent Conditioned Loco-Manipulation Using Motion Priorscited by 3
  38. 2026Learning Human-Like Badminton Skills for Humanoid Robotscited by 3

50 most cited of 89 All 89 (JSON)