# Learning Deployable Locomotion Control via Differentiable Simulation

arXiv: https://arxiv.org/abs/2404.02887 · HTML: https://mlsanity.com/paper/2404.02887/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Apr 2024.

## Built on by (most cited first, 8 of 8)

- [Stabilizing Reinforcement Learning in Differentiable Multiphysics Simulation](https://mlsanity.com/paper/2412.12089/) (2024)
- [Reparameterization Flow Policy Optimization](https://mlsanity.com/paper/2602.03501/) (2026)
- [DiffSim2Real: Deploying Quadrupedal Locomotion Policies Purely Trained in Differentiable Simulation](https://mlsanity.com/paper/2411.02189/) (2024)
- [Efficient On-policy Visual-RL via Stochastic Decoupled Policy Gradient](https://mlsanity.com/paper/2605.26478/) (2026)
- [Where-to-Learn: Analytical Policy Gradient Directed Exploration for On-Policy Robotic Reinforcement Learning](https://mlsanity.com/paper/2603.27317/) (2026)
- [Tempered Sequential Monte Carlo for Trajectory and Policy Optimization with Differentiable Dynamics](https://mlsanity.com/paper/2604.21456/) (2026)
- [Open-DiffLoco: Open-Source Differentiable Learning for Deployable Blind Quadruped Locomotion](https://mlsanity.com/paper/2608.02069/) (2026)
- [E2DRL: End-to-End Differentiable Reinforcement Learning for 3D Obstacle Avoidance in Quadruped Robots](https://mlsanity.com/paper/2610.19627/) (2026)

## Abstract (arXiv)

Differentiable simulators promise to improve sample efficiency in robot learning by providing analytic gradients of the system dynamics. Yet, their application to contact-rich tasks like locomotion is complicated by the inherently non-smooth nature of contact, impeding effective gradient-based optimization. Existing works thus often rely on soft contact models that provide smooth gradients but lack physical accuracy, constraining results to simulation. To address this limitation, we propose a differentiable contact model designed to provide informative gradients while maintaining high physical fidelity. We demonstrate the efficacy of our approach by training a quadrupedal locomotion policy within our differentiable simulator leveraging analytic gradients and successfully transferring the learned policy zero-shot to the real world. To the best of our knowledge, this represents the first successful sim-to-real transfer of a legged locomotion policy learned entirely within a differentiable simulator, establishing the feasibility of using differentiable simulation for real-world locomotion control.