# Dynamics Distillation for Efficient and Transferable Control Learning

arXiv: https://arxiv.org/abs/2605.01516 · HTML: https://mlsanity.com/paper/2605.01516/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods.
- Builds on 29 of its 46 references that are in the map.
- First posted on arXiv 2 May 2026.

## Placed in

- Deep reinforcement learning methods / [RL for autonomous driving and traffic](https://mlsanity.com/deeprl/rl-for-autonomous-driving-and-traffic/)

## Builds on (most cited first, 29 of 29)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset](https://mlsanity.com/paper/2104.10133/) (2021)
- [MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning](https://mlsanity.com/paper/2109.12674/) (2021)
- [EMMA: End-to-End Multimodal Model for Autonomous Driving](https://mlsanity.com/paper/2410.23262/) (2024)
- [Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research](https://mlsanity.com/paper/2310.08710/) (2023)
- [Pseudo-Simulation for Autonomous Driving](https://mlsanity.com/paper/2506.04218/) (2025)
- [Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail](https://mlsanity.com/paper/2511.00088/) (2025)
- [Baidu Apollo EM Motion Planner](https://mlsanity.com/paper/1807.08048/) (2018)
- [Robust Autonomy Emerges from Self-Play](https://mlsanity.com/paper/2502.03349/) (2025)
- [WOD-E2E: Waymo Open Dataset for End-to-End Driving in Challenging Long-tail Scenarios](https://mlsanity.com/paper/2510.26125/) (2025)
- [Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics](https://mlsanity.com/paper/2501.10100/) (2025)
- [GPUDrive: Data-driven, multi-agent driving simulation at 1 million FPS](https://mlsanity.com/paper/2408.01584/) (2024)
- [CaRL: Learning Scalable Planning Policies with Simple Rewards](https://mlsanity.com/paper/2504.17838/) (2025)
- [Producing and Leveraging Online Map Uncertainty in Trajectory Prediction](https://mlsanity.com/paper/2403.16439/) (2024)
- [High-speed Autonomous Drifting with Deep Reinforcement Learning](https://mlsanity.com/paper/2001.01377/) (2020)
- [LocoFormer: Generalist Locomotion via Long-context Adaptation](https://mlsanity.com/paper/2509.23745/) (2025)
- [Data Scaling Laws for End-to-End Autonomous Driving](https://mlsanity.com/paper/2504.04338/) (2025)
- [AnyCar to Anywhere: Learning Universal Dynamics Model for Agile and Adaptive Mobility](https://mlsanity.com/paper/2409.15783/) (2024)
- [RACECAR -- The Dataset for High-Speed Autonomous Racing](https://mlsanity.com/paper/2306.03252/) (2023)
- [Deep Dynamics: Vehicle Dynamics Modeling with a Physics-Constrained Neural Network for Autonomous Racing](https://mlsanity.com/paper/2312.04374/) (2023)
- [RAPTOR: A Foundation Policy for Quadrotor Control](https://mlsanity.com/paper/2509.11481/) (2025)
- [A Simulation Benchmark for Autonomous Racing with Large-Scale Human Data](https://mlsanity.com/paper/2407.16680/) (2024)
- [A Sequential Two-Step Algorithm for Fast Generation of Vehicle Racing Trajectories](https://mlsanity.com/paper/1902.00606/) (2019)
- [Hybrid Physics and Deep Learning Model for Interpretable Vehicle State Prediction](https://mlsanity.com/paper/2103.06727/) (2021)
- [Deep Reinforcement Learning in Autonomous Car Path Planning and Control: A Survey](https://mlsanity.com/paper/2404.00340/) (2024)
- [Out-of-Distribution Generalization with a SPARC: Racing 100 Unseen Vehicles with a Single Policy](https://mlsanity.com/paper/2511.09737/) (2025)
- [Residual Learning towards High-fidelity Vehicle Dynamics Modeling with Transformer](https://mlsanity.com/paper/2502.11800/) (2025)

## Abstract (arXiv)

Robust control policy learning for autonomous driving requires training environments to be both physically realistic and computationally scalable, properties that existing simulators provide only in isolation. We introduce Sim2Sim2Sim, a framework that bridges high-fidelity vehicle simulation and scalable reinforcement learning by distilling simulator dynamics into a highly parallelizable learned dynamics model. By training control policies purely within this distilled environment and deploying them back into the high-fidelity source simulator, we demonstrate more efficient policy optimization and reliable transfer under challenging dynamics. We further show that predictive accuracy alone does not fully characterize a learned dynamics model's suitability as a reinforcement learning training environment, which should also be assessed by the quality of the policies it enables.