mlsanityData to 7 Oct 2026

Latent Imagination Facilitates Zero-Shot Transfer in Autonomous Racing

arXiv 2103.04909 ↗First posted 8 Mar 2021cs.LG cs.AI cs.NEcited by 22

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

World models learn behaviors in a latent imagination space to enhance the sample-efficiency of deep reinforcement learning (RL) algorithms. While learning world models for high-dimensional observations (e.g., pixel inputs) has become practicable on standard RL benchmarks and some games, their effectiveness in real-world robotics applications has not been explored. In this paper, we investigate how such agents generalize to real-world autonomous vehicle control tasks, where advanced model-free deep RL algorithms fail. In particular, we set up a series of time-lap tasks for an F1TENTH racing robot, equipped with a high-dimensional LiDAR sensor, on a set of test tracks with a gradual increase in their complexity. In this continuous-control setting, we show that model-based agents capable of learning in imagination substantially outperform model-free agents with respect to performance, sample efficiency, successful task completion, and generalization. Moreover, we show that the generalization ability of model-based agents strongly depends on the choice of their observation model. We provide extensive empirical evidence for the effectiveness of world models provided with long enough memory horizons in sim2real tasks.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 28 of its 63 references that are in the map. First posted on arXiv 8 Mar 2021.

Builds on

28 of 63 references.

Closest ancestors · 3

Same direction, newest first

  1. 2020Mastering Atari with Discrete World Models5 Oct
  2. 2019Dream to Control: Learning Behaviors by Latent Imagination3 Dec
  3. 2018Recurrent World Models Facilitate Policy Evolution4 Sep

Other references · 25

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  4. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
  5. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  6. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  7. 2018Maximum a Posteriori Policy Optimisationcited by 318
  8. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 277
  9. 2018Learning to Drive in a Daycited by 266
  10. 2018Distributed Distributional Deterministic Policy Gradientscited by 195
  11. 2020Liquid Time-constant Networkscited by 160
  12. 2018Learning to Walk via Deep Reinforcement Learningcited by 153
13 more
  1. 2019End-to-End Robotic Reinforcement Learning without Reward Engineeringcited by 115
  2. 2019Imitation Learning from Imperfect Demonstrationcited by 112
  3. 2020The Ingredients of Real-World Robotic Reinforcement Learningcited by 111
  4. 2020Learning Long-Term Dependencies in Irregularly-Sampled Time Seriescited by 82
  5. 2020Super-Human Performance in Gran Turismo Sport Using Deep Reinforcement Learningcited by 60
  6. 2020Regularizing Action Policies for Smooth Control with Reinforcement Learningcited by 47
  7. 2020Sim2Real Transfer for Reinforcement Learning without Dynamics Randomizationcited by 43
  8. 2018End-to-End Race Driving with Deep Reinforcement Learningcited by 38
  9. 2019AMZ Driverless: The Full Autonomous Racing Systemcited by 28
  10. 2021Deep Latent Competition: Learning to Race Using Visual Control Policies in Latent Spacecited by 20
  11. 2019Adversarial Imitation Learning from Incomplete Demonstrationscited by 16
  12. 2020Optimization-Based Hierarchical Motion Planning for Autonomous Racingcited by 11
  13. 2018Response Characterization for Auditing Cell Dynamics in Long Short-term Memory Networkscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 5; 2023: 6; 2024: 4; 2025: 4; 2026: 1.

22 papers.

Sort:

By area:

  1. 2021Closed-form Continuous-time Neural Modelscited by 65
  2. 2023TorchRL: A data-driven decision-making library for PyTorchcited by 59
  3. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Surveycited by 30
  4. 2023TWIST: Teacher-Student World Model Distillation for Efficient Sim-to-Real Transfercited by 18
  5. 2025Dream to Drive with Predictive Individual World Modelcited by 11
  6. 2022Differentiable Control Barrier Functions for Vision-based End-to-End Autonomous Drivingcited by 10
  7. 2025Accelerating Model-Based Reinforcement Learning with State-Space World Modelscited by 10
  8. 2021Hierarchical Potential-based Reward Shaping from Task Specificationscited by 7
  9. 2022TC-Driver: Trajectory Conditioned Driving for Robust Autonomous Racing -- A Reinforcement Learning Approachcited by 6
  10. 2024Autonomous Driving Small-Scale Cars: A Survey of Recent Developmentcited by 6
  11. 2022Are All Vision Models Created Equal? A Study of the Open-Loop to Closed-Loop Causality Gapcited by 5
  12. 2023Residual Policy Learning for Vehicle Control of Autonomous Racing Carscited by 4
10 more
  1. 2023A Mathematical Characterization of Minimally Sufficient Robot Brainscited by 4
  2. 2025RLPP: A Residual Method for Zero-Shot Real-World Autonomous Racing on Scaled Platformscited by 2
  3. 2022Interpreting Neural Policies with Disentangled Tree Representationscited by 1
  4. 2023Sense, Imagine, Act: Multimodal Perception Improves Model-Based Reinforcement Learning for Head-to-Head Autonomous Racingcited by 1
  5. 2024DODT: Enhanced Online Decision Transformer Learning through Dreamer's Actor-Critic Trajectory Forecastingcited by 1
  6. 2022Safe Policy Improvement in Constrained Markov Decision Processescited by 0
  7. 2023Enhancing Robot Learning through Learned Human-Attention Feature Mapscited by 0
  8. 2024F1tenth Autonomous Racing With Offline Reinforcement Learning Methodscited by 0
  9. 2025Context-Aware Model-Based Reinforcement Learning for Autonomous Racingcited by 0
  10. 2026Autonomous Vehicle Collision Avoidance With Racing Parameterized Deep Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021Cycle-Consistent World Models for Domain Independent Latent Imagination
  2. 2026DreamerAD: Efficient Reinforcement Learning via Latent World Model for Autonomous Driving
  3. 2023Model-Based Reinforcement Learning with Isolated Imaginations
  4. 2025DREAMer-VXS: A Latent World Model for Sample-Efficient AGV Exploration in Stochastic, Unobserved Environments
  5. 2024Open-World Reinforcement Learning over Long Short-Term Imagination
  6. 2025WMPO: World Model-based Policy Optimization for Vision-Language-Action Models
  7. 2026World Model for Robot Learning: A Comprehensive Survey
  8. 2025Adapting World Models with Latent-State Dynamics Residuals