mlsanityData to 7 Oct 2026

Hierarchical Model-Based Imitation Learning for Planning in Autonomous Driving

arXiv 2210.09539 ↗First posted 18 Oct 2022cs.RO cs.AI cs.LGcited by 43

On the map: End-to-end autonomous driving → Imitation-learning end-to-end planners

also in 1 more
Abstract (arXiv)

We demonstrate the first large-scale application of model-based generative adversarial imitation learning (MGAIL) to the task of dense urban self-driving. We augment standard MGAIL using a hierarchical model to enable generalization to arbitrary goal routes, and measure performance using a closed-loop evaluation framework with simulated interactive agents. We train policies from expert trajectories collected from real vehicles driving over 100,000 miles in San Francisco, and demonstrate a steerable policy that can navigate robustly even in a zero-shot setting, generalizing to synthetic scenarios with novel goals that never occurred in real-world driving. We also demonstrate the importance of mixing closed-loop MGAIL losses with open-loop behavior cloning losses, and show our best policy approaches the performance of the expert. We evaluate our imitative model in both average and challenging scenarios, and show how it can serve as a useful prior to plan successful trajectories.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving. Builds on 47 of its 60 references that are in the map. First posted on arXiv 18 Oct 2022.

Builds on

47 of 60 references.

References · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2016Layer Normalizationcited by 5,579
  4. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  5. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  6. 2016Concrete Problems in AI Safetycited by 1,582
  7. 2016Generative Adversarial Imitation Learningcited by 1,576
  8. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
  9. 2021Perceiver: General Perception with Iterative Attentioncited by 1,089
  10. 2016End to End Learning for Self-Driving Carscited by 1,000
  11. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 532
  12. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
35 more
  1. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 485
  2. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  3. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  4. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
  5. 2019Learning by Cheatingcited by 361
  6. 2019Causal Confusion in Imitation Learningcited by 286
  7. 2020Integrated Task and Motion Planningcited by 281
  8. 2018Learning to Drive in a Daycited by 266
  9. 2016A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehiclescited by 251
  10. 2021MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Predictioncited by 246
  11. 2021End-to-end Interpretable Neural Motion Plannercited by 240
  12. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 211
  13. 2021Multimodal Motion Prediction with Stacked Transformerscited by 207
  14. 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 192
  15. 2018Generative Adversarial Imitation from Observationcited by 183
  16. 2019PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settingscited by 178
  17. 2021Scene Transformer: A unified architecture for predicting multiple agent trajectoriescited by 165
  18. 2019Goal-conditioned Imitation Learningcited by 159
  19. 2018Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networkscited by 129
  20. 2020SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Drivingcited by 121
  21. 2021Reward (Mis)design for Autonomous Drivingcited by 102
  22. 2018Driving Policy Transfer via Modularity and Abstractioncited by 91
  23. 2019Multi-Head Attention for Multi-Modal Joint Vehicle Motion Forecastingcited by 87
  24. 2019IRIS: Implicit Reinforcement without Interaction at Scale for Learning Control from Offline Robot Manipulation Datacited by 85
  25. 2021Of Moments and Matching: A Game-Theoretic Framework for Closing the Imitation Gapcited by 80
  26. 2017Virtual to Real Reinforcement Learning for Autonomous Drivingcited by 79
  27. 2018Scalable End-to-End Autonomous Vehicle Testing via Rare-event Simulationcited by 76
  28. 2018Deep Imitative Models for Flexible Inference, Planning, and Controlcited by 75
  29. 2021SafetyNet: Safe planning for real-world self-driving vehicles using machine-learned policiescited by 68
  30. 2021Shaking the foundations: delusions in sequence models for interaction and controlcited by 61
  31. 2022Symphony: Learning Realistic and Diverse Agents for Autonomous Driving Simulationcited by 61
  32. 2018Variance reduction properties of the reparameterization trickcited by 41
  33. 2021Autonomy 2.0: Why is self-driving always 5 years away?cited by 30
  34. 2018Learning from Demonstration in the Wildcited by 25
  35. 2021Deep Structured Reactive Planningcited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 16; 2024: 11; 2025: 9; 2026: 4.

43 papers.

Sort:

By area:

  1. 2023Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Researchcited by 158
  2. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarioscited by 124
  3. 2023PillarNeXt: Rethinking Network Designs for 3D Object Detection in LiDAR Point Cloudscited by 54
  4. 2024GPUDrive: Data-driven, multi-agent driving simulation at 1 million FPScited by 43
  5. 2023DistillBEV: Boosting Multi-Camera 3D Object Detection with Cross-Modal Knowledge Distillationcited by 41
  6. 2023Inverse Reinforcement Learning without Reinforcement Learningcited by 35
  7. 2024Hybrid Inverse Reinforcement Learningcited by 31
  8. 2023The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Reviewcited by 29
  9. 2025The Role of World Models in Shaping Autonomous Driving: A Comprehensive Surveycited by 29
  10. 2023Rethinking Imitation-based Planner for Autonomous Drivingcited by 25
  11. 2023Machine Learning for Autonomous Vehicle's Trajectory Prediction: A comprehensive survey, Challenges, and Future Research Directionscited by 23
  12. 2022Embedding Synthetic Off-Policy Experience for Autonomous Driving via Zero-Shot Curriculacited by 17
31 more
  1. 2023Boosting Offline Reinforcement Learning for Autonomous Driving with Hierarchical Latent Skillscited by 13
  2. 2024MOTO: Offline Pre-training to Online Fine-tuning for Model-based Robot Learningcited by 13
  3. 2023Hierarchical Generative Adversarial Imitation Learning with Mid-level Input Generation for Autonomous Driving on Urban Environmentscited by 11
  4. 2025Dream to Drive with Predictive Individual World Modelcited by 11
  5. 2024Non-Adversarial Inverse Reinforcement Learning via Successor Feature Matchingcited by 10
  6. 2023Transcendental Idealism of Planner: Evaluating Perception from Planning Perspective for Autonomous Drivingcited by 9
  7. 2023Uncertainty-Aware Decision Transformer for Stochastic Driving Environmentscited by 8
  8. 2023Selective Communication for Cooperative Perception in End-to-End Autonomous Drivingcited by 5
  9. 2024Adversarial Imitation Learning via Boostingcited by 5
  10. 2022Evaluation of MPC-based Imitation Learning for Human-like Autonomous Drivingcited by 4
  11. 2023Interpretable Motion Planner for Urban Driving via Hierarchical Imitation Learningcited by 4
  12. 2024Validity Learning on Failures: Mitigating the Distribution Shift in Autonomous Vehicle Planningcited by 4
  13. 2024Dynamic High-Order Control Barrier Functions with Diffuser for Safety-Critical Trajectory Planning at Signal-Free Intersectionscited by 4
  14. 2025SafePath: Conformal Prediction for Safe LLM-Based Autonomous Navigationcited by 4
  15. 2023Striking a Balance: An Optimal Mechanism Design for Heterogenous Differentially Private Data Acquisition for Logistic Regressioncited by 3
  16. 2025V-Max: A Reinforcement Learning Framework for Autonomous Drivingcited by 3
  17. 2025MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous drivingcited by 3
  18. 2023A Survey on Datasets for Decision-making of Autonomous Vehiclecited by 2
  19. 2023Autonomous Driving using Residual Sensor Fusion and Deep Reinforcement Learningcited by 2
  20. 2025Model Predictive Adversarial Imitation Learning for Planning from Observationcited by 2
  21. 2024Analyzing Closed-loop Training Techniques for Realistic Traffic Agent Models in Autonomous Highway Driving Simulationscited by 1
  22. 2026D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Graspingcited by 1
  23. 2026Zero-shot Imitation Learning by Latent Topology Mappingcited by 1
  24. 2024Offline Imitation of Badminton Player Behavior via Experiential Contexts and Brownian Motioncited by 0
  25. 2024MRIC: Model-Based Reinforcement-Imitation Learning with Mixture-of-Codebooks for Autonomous Driving Simulationcited by 0
  26. 2024Vectorized Representation Dreamer (VRD): Dreaming-Assisted Multi-Agent Motion-Forecastingcited by 0
  27. 2025Systems Engineering for Autonomous Vehicles; Supervising AI using Large Language Models (SSuperLLM)cited by 0
  28. 2025CAPS: Context-Aware Priority Sampling for Enhanced Imitation Learning in Autonomous Drivingcited by 0
  29. 2025Coupled Distributional Random Expert Distillation for World Model Online Imitation Learningcited by 0
  30. 2026Artificial Intelligence for Modeling and Simulation of Mixed Automated and Human Trafficcited by 0
  31. 2026Reward-Conditioned Attention: How Reward Design Shapes What Autonomous Driving Agents Seecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022SHAIL: Safety-Aware Hierarchical Adversarial Imitation Learning for Autonomous Driving in Urban Environments
  2. 2021Generative Adversarial Imitation Learning for End-to-End Autonomous Driving on Urban Environments
  3. 2020Modeling Human Driving Behavior through Generative Adversarial Imitation Learning
  4. 2018Multi-Agent Imitation Learning for Driving Simulation
  5. 2022Model-Based Imitation Learning for Urban Driving
  6. 2024PLUTO: Pushing the Limit of Imitation Learning-based Planning for Autonomous Driving
  7. 2025Exposing the Copycat Problem of Imitation-based Planner: A Novel Closed-Loop Simulator, Causal Benchmark and Joint IL-RL Baseline
  8. 2025Sequence of Expert: Boosting Imitation Planners for Autonomous Driving through Temporal Alternation