# CARL: Controllable Agent with Reinforcement Learning for Quadruped Locomotion

arXiv: https://arxiv.org/abs/2005.03288 · HTML: https://mlsanity.com/paper/2005.03288/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 May 2020.

## Built on by (most cited first, 7 of 7)

- [Learning to Brachiate via Simplified Model Imitation](https://mlsanity.com/paper/2205.03943/) (2022)
- [An Adaptable Approach to Learn Realistic Legged Locomotion without Examples](https://mlsanity.com/paper/2110.14998/) (2021)
- [Bidirectional GaitNet: A Bidirectional Prediction Model of Human Gait and Anatomical Conditions](https://mlsanity.com/paper/2306.04161/) (2023)
- [Quadruped Locomotion on Non-Rigid Terrain using Reinforcement Learning](https://mlsanity.com/paper/2107.02955/) (2021)
- [Transition Motion Tensor: A Data-Driven Approach for Versatile and Controllable Agents in Physically Simulated Environments](https://mlsanity.com/paper/2111.15072/) (2021)
- [Open-Ended Learning Strategies for Learning Complex Locomotion Skills](https://mlsanity.com/paper/2206.06796/) (2022)
- [Expanding Versatility of Agile Locomotion through Policy Transitions Using Latent State Representation](https://mlsanity.com/paper/2306.08224/) (2023)

## Abstract (arXiv)

Motion synthesis in a dynamic environment has been a long-standing problem for character animation. Methods using motion capture data tend to scale poorly in complex environments because of their larger capturing and labeling requirement. Physics-based controllers are effective in this regard, albeit less controllable. In this paper, we present CARL, a quadruped agent that can be controlled with high-level directives and react naturally to dynamic environments. Starting with an agent that can imitate individual animation clips, we use Generative Adversarial Networks to adapt high-level controls, such as speed and heading, to action distributions that correspond to the original animations. Further fine-tuning through the deep reinforcement learning enables the agent to recover from unseen external perturbations while producing smooth transitions. It then becomes straightforward to create autonomous agents in dynamic environments by adding navigation modules over the entire process. We evaluate our approach by measuring the agent's ability to follow user control and provide a visual analysis of the generated motion to show its effectiveness.