mlsanityData to 14 Oct 2026

Dynamics Distillation for Efficient and Transferable Control Learning

arXiv 2605.01516 ↗First posted 2 May 2026cs.ROcited by 0

On the map: Deep reinforcement learning methods → RL for autonomous driving and traffic

Abstract (arXiv)

Robust control policy learning for autonomous driving requires training environments to be both physically realistic and computationally scalable, properties that existing simulators provide only in isolation. We introduce Sim2Sim2Sim, a framework that bridges high-fidelity vehicle simulation and scalable reinforcement learning by distilling simulator dynamics into a highly parallelizable learned dynamics model. By training control policies purely within this distilled environment and deploying them back into the high-fidelity source simulator, we demonstrate more efficient policy optimization and reliable transfer under challenging dynamics. We further show that predictive accuracy alone does not fully characterize a learned dynamics model's suitability as a reinforcement learning training environment, which should also be assessed by the quality of the policies it enables.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods. Builds on 29 of its 46 references that are in the map. First posted on arXiv 2 May 2026.

Builds on

29 of 46 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025CaRL: Learning Scalable Planning Policies with Simple Rewards24 Apr
  2. 2024Deep Reinforcement Learning in Autonomous Car Path Planning and Control: A Survey30 Mar

Other references · 27

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  4. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 537
  5. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 290
  6. 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 231
  7. 2023Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Researchcited by 160
  8. 2025Pseudo-Simulation for Autonomous Drivingcited by 150
  9. 2025Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tailcited by 146
  10. 2018Baidu Apollo EM Motion Plannercited by 100
  11. 2025Robust Autonomy Emerges from Self-Playcited by 67
  12. 2025WOD-E2E: Waymo Open Dataset for End-to-End Driving in Challenging Long-tail Scenarioscited by 62
15 more
  1. 2025Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Roboticscited by 51
  2. 2024GPUDrive: Data-driven, multi-agent driving simulation at 1 million FPScited by 45
  3. 2024Producing and Leveraging Online Map Uncertainty in Trajectory Predictioncited by 33
  4. 2020High-speed Autonomous Drifting with Deep Reinforcement Learningcited by 24
  5. 2025LocoFormer: Generalist Locomotion via Long-context Adaptationcited by 20
  6. 2025Data Scaling Laws for End-to-End Autonomous Drivingcited by 15
  7. 2024AnyCar to Anywhere: Learning Universal Dynamics Model for Agile and Adaptive Mobilitycited by 13
  8. 2023RACECAR -- The Dataset for High-Speed Autonomous Racingcited by 9
  9. 2023Deep Dynamics: Vehicle Dynamics Modeling with a Physics-Constrained Neural Network for Autonomous Racingcited by 8
  10. 2025RAPTOR: A Foundation Policy for Quadrotor Controlcited by 8
  11. 2024A Simulation Benchmark for Autonomous Racing with Large-Scale Human Datacited by 6
  12. 2019A Sequential Two-Step Algorithm for Fast Generation of Vehicle Racing Trajectoriescited by 5
  13. 2021Hybrid Physics and Deep Learning Model for Interpretable Vehicle State Predictioncited by 5
  14. 2025Out-of-Distribution Generalization with a SPARC: Racing 100 Unseen Vehicles with a Single Policycited by 5
  15. 2025Residual Learning towards High-fidelity Vehicle Dynamics Modeling with Transformercited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Scaling Curriculum Learning For Autonomous Driving
  2. 2026RAPiD: Reward-Guided Consistency Distillation of Diffusion Planners for Real-Time Autonomous Driving
  3. 2025Automatic Curriculum Learning for Driving Scenarios: Towards Robust and Efficient Reinforcement Learning
  4. 2022Discrete Control in Real-World Driving Environments using Deep Reinforcement Learning
  5. 2024Knowledge Transfer from Simple to Complex: A Safe and Efficient Reinforcement Learning Framework for Autonomous Driving Decision-Making
  6. 2025Multi-Timescale Hierarchical Reinforcement Learning for Unified Behavior and Control of Autonomous Driving
  7. 2026Sim2Real-AD: A Modular Sim-to-Real Framework for Deploying VLM-Guided Reinforcement Learning in Real-World Autonomous Driving
  8. 2025Controllable Latent Diffusion for Traffic Simulation