mlsanityData to 7 Oct 2026

Embedding Synthetic Off-Policy Experience for Autonomous Driving via Zero-Shot Curricula

arXiv 2212.01375 ↗First posted 2 Dec 2022cs.RO cs.AI cs.LGcited by 17

On the map: End-to-end autonomous driving → Imitation-learning end-to-end planners

Abstract (arXiv)

ML-based motion planning is a promising approach to produce agents that exhibit complex behaviors, and automatically adapt to novel environments. In the context of autonomous driving, it is common to treat all available training data equally. However, this approach produces agents that do not perform robustly in safety-critical settings, an issue that cannot be addressed by simply adding more data to the training set - we show that an agent trained using only a 10% subset of the data performs just as well as an agent trained on the entire dataset. We present a method to predict the inherent difficulty of a driving situation given data collected from a fleet of autonomous vehicles deployed on public roads. We then demonstrate that this difficulty score can be used in a zero-shot transfer to generate curricula for an imitation-learning based planning agent. Compared to training on the entire unbiased training dataset, we show that prioritizing difficult driving scenarios both reduces collisions by 15% and increases route adherence by 14% in closed-loop evaluation, all while using only 10% of the training data.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving. Builds on 26 of its 47 references that are in the map. First posted on arXiv 2 Dec 2022.

Builds on

26 of 47 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Hierarchical Model-Based Imitation Learning for Planning in Autonomous Driving18 Oct

Other references · 25

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  3. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  4. 2016Generative Adversarial Imitation Learningcited by 1,576
  5. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  6. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  7. 2021Perceiver: General Perception with Iterative Attentioncited by 1,089
  8. 2015Prioritized Experience Replaycited by 1,070
  9. 2019Solving Rubik's Cube with a Robot Handcited by 699
  10. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  11. 2016Training Region-based Object Detectors with Online Hard Example Miningcited by 577
  12. 2021Implicit Behavioral Cloningcited by 442
13 more
  1. 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 301
  2. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 256
  3. 2021Multimodal Motion Prediction with Stacked Transformerscited by 207
  4. 2018Generative Adversarial Imitation from Observationcited by 183
  5. 2020Automatic Curriculum Learning For Deep RL: A Short Surveycited by 141
  6. 2018Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networkscited by 129
  7. 2019Multi-Head Attention for Multi-Modal Joint Vehicle Motion Forecastingcited by 87
  8. 2022When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning?cited by 86
  9. 2021Of Moments and Matching: A Game-Theoretic Framework for Closing the Imitation Gapcited by 80
  10. 2018CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learningcited by 78
  11. 2018Learning Cross-Lingual Sentence Representations via a Multi-task Dual-Encoder Modelcited by 46
  12. 2018Variance reduction properties of the reparameterization trickcited by 41
  13. 2018Fingerprint Policy Optimisation for Robust Reinforcement Learningcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 2; 2024: 6; 2025: 6; 2026: 2.

17 papers.

Sort:

By area:

  1. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarioscited by 124
  2. 2023Learning Realistic Traffic Agents in Closed-loopcited by 24
  3. 2024CaDRE: Controllable and Diverse Generation of Safety-Critical Driving Scenarios using Real-World Trajectoriescited by 11
  4. 2024Learning to Drive via Asymmetric Self-Playcited by 11
  5. 2024LoRD: Adapting Differentiable Driving Policies to Distribution Shiftscited by 4
  6. 2025V-Max: A Reinforcement Learning Framework for Autonomous Drivingcited by 3
  7. 2023Scaling Is All You Need: Autonomous Driving with JAX-Accelerated Reinforcement Learningcited by 2
  8. 2024Rate-Informed Discovery via Bayesian Adaptive Multifidelity Samplingcited by 2
  9. 2024AdaDemo: Data-Efficient Demonstration Expansion for Generalist Robotic Agentcited by 1
  10. 2025Synthetic Data for Robust AI Model Development in Regulated Enterprisescited by 1
  11. 2024Foundation Models for Rapid Autonomy Validationcited by 0
  12. 2025Systems Engineering for Autonomous Vehicles; Supervising AI using Large Language Models (SSuperLLM)cited by 0
5 more
  1. 2025What should an AI assessor optimise for?cited by 0
  2. 2025CAPS: Context-Aware Priority Sampling for Enhanced Imitation Learning in Autonomous Drivingcited by 0
  3. 2025Learning to Drive by Imitating Surrounding Vehiclescited by 0
  4. 2026From Attacks to Curricula: Learnability-Guided Adversarial Training for Safe Autonomous Drivingcited by 0
  5. 2026Large Language Model Enhanced Differentiable Trajectory Planning for IoT-Enabled Autonomous Drivingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Learning from Demonstrations of Critical Driving Behaviours Using Driver's Risk Field
  2. 2025Automatic Curriculum Learning for Driving Scenarios: Towards Robust and Efficient Reinforcement Learning
  3. 2025From Imitation to Optimization: A Comparative Study of Offline Learning for Autonomous Driving
  4. 2026Scaling Curriculum Learning For Autonomous Driving
  5. 2022Distribution-aware Goal Prediction and Conformant Model-based Planning for Safe Autonomous Driving
  6. 2024Validity Learning on Failures: Mitigating the Distribution Shift in Autonomous Vehicle Planning
  7. 2022SHAIL: Safety-Aware Hierarchical Adversarial Imitation Learning for Autonomous Driving in Urban Environments
  8. 2025Exposing the Copycat Problem of Imitation-based Planner: A Novel Closed-Loop Simulator, Causal Benchmark and Joint IL-RL Baseline