# Hierarchical Generative Adversarial Imitation Learning with Mid-level Input Generation for Autonomous Driving on Urban Environments

arXiv: https://arxiv.org/abs/2302.04823 · HTML: https://mlsanity.com/paper/2302.04823/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Adversarial scenarios and safety testing, one of 9 directions in End-to-end autonomous driving.
- Builds on 29 of its 49 references that are in the map.
- First posted on arXiv 9 Feb 2023.

## Placed in

- End-to-end autonomous driving / [Adversarial scenarios and safety testing](https://mlsanity.com/driving/adversarial-scenarios-and-safety-testing/)
- Offline RL and imitation learning from data / [Imitation and offline RL for driving](https://mlsanity.com/offlinerl/imitation-and-offline-rl-for-driving/)

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [End-to-end Driving via Conditional Imitation Learning](https://mlsanity.com/paper/1710.02410/) (2017)
- [ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst](https://mlsanity.com/paper/1812.03079/) (2018)
- [Learning by Cheating](https://mlsanity.com/paper/1912.12294/) (2019)
- [Exploring the Limitations of Behavior Cloning for Autonomous Driving](https://mlsanity.com/paper/1904.08980/) (2019)
- [Decoupling Representation Learning from Reinforcement Learning](https://mlsanity.com/paper/2009.08319/) (2020)
- [A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehicles](https://mlsanity.com/paper/1604.07446/) (2016)
- [End-to-End Urban Driving by Imitating a Reinforcement Learning Coach](https://mlsanity.com/paper/2108.08265/) (2021)
- [End-to-end Learning of Driving Models from Large-scale Video Datasets](https://mlsanity.com/paper/1612.01079/) (2016)
- [Visual Semantic Navigation using Scene Priors](https://mlsanity.com/paper/1810.06543/) (2018)
- [Imitating Driver Behavior with Generative Adversarial Networks](https://mlsanity.com/paper/1701.06699/) (2017)
- [Driving Policy Transfer via Modularity and Abstraction](https://mlsanity.com/paper/1804.09364/) (2018)
- [Visual Representations for Semantic Target Driven Navigation](https://mlsanity.com/paper/1805.06066/) (2018)
- [Hierarchical Model-Based Imitation Learning for Planning in Autonomous Driving](https://mlsanity.com/paper/2210.09539/) (2022)
- [Training Larger Networks for Deep Reinforcement Learning](https://mlsanity.com/paper/2102.07920/) (2021)
- [Learning to Navigate Using Mid-Level Visual Priors](https://mlsanity.com/paper/1912.11121/) (2019)
- [Probabilistic End-to-End Vehicle Navigation in Complex Dynamic Environments with Multimodal Sensor Fusion](https://mlsanity.com/paper/2005.01935/) (2020)
- [Vision-Based Autonomous Car Racing Using Deep Imitative Reinforcement Learning](https://mlsanity.com/paper/2107.08325/) (2021)
- [Wasserstein Distance guided Adversarial Imitation Learning with Reward Shape Exploration](https://mlsanity.com/paper/2006.03503/) (2020)
- [Proximal Policy Optimization with Continuous Bounded Action Space via the Beta Distribution](https://mlsanity.com/paper/2111.02202/) (2021)
- [Imitation Learning of Hierarchical Driving Model: from Continuous Intention to Continuous Trajectory](https://mlsanity.com/paper/2010.10393/) (2020)
- [Generative Adversarial Imitation Learning for End-to-End Autonomous Driving on Urban Environments](https://mlsanity.com/paper/2110.08586/) (2021)

## Built on by (most cited first, 12 of 12)

- [GaussianAD: Gaussian-Centric End-to-End Autonomous Driving](https://mlsanity.com/paper/2412.10371/) (2024)
- [Rethinking Imitation-based Planner for Autonomous Driving](https://mlsanity.com/paper/2309.10443/) (2023)
- [A Comprehensive Review of Reinforcement Learning for Autonomous Driving in the CARLA Simulator](https://mlsanity.com/paper/2509.08221/) (2025)
- [Hierarchical end-to-end autonomous navigation through few-shot waypoint detection](https://mlsanity.com/paper/2409.14633/) (2024)
- [An Initial Study of Bird's-Eye View Generation for Autonomous Vehicles using Cross-View Transformers](https://mlsanity.com/paper/2508.12520/) (2025)
- [Exploring multimodal implicit behavior learning for vehicle navigation in simulated cities](https://mlsanity.com/paper/2509.15400/) (2025)
- [Towards Physics-informed Spatial Intelligence with Human Priors: An Autonomous Driving Pilot Study](https://mlsanity.com/paper/2510.21160/) (2025)
- [Closed-Loop Evaluation of Bird's-Eye-View Maps from Cross-View Transformers as Inputs to Behavior-Cloning Policies](https://mlsanity.com/paper/2609.05783/) (2026)
- [Disagreement-Regularized Imitation Learning for Image-Based Continuous Control with Gaussian and Beta Policies](https://mlsanity.com/paper/2609.38407/) (2026)
- [Diffusion-2BC: Hybrid Diffusion and Regression Training for Offline Behavior Cloning in Autonomous Driving](https://mlsanity.com/paper/2609.38472/) (2026)
- [A Survey on End-to-End Autonomous Driving Training from the Perspectives of Data, Strategy, and Platform](https://mlsanity.com/paper/2610.00926/) (2026)
- [Investigating Behavior Cloning from Few Demonstrations for Autonomous Driving based on Bird's-Eye View in Simulated Cities](https://mlsanity.com/paper/2610.20203/) (2026)

## Abstract (arXiv)

Deriving robust control policies for realistic urban navigation scenarios is not a trivial task. In an end-to-end approach, these policies must map high-dimensional images from the vehicle's cameras to low-level actions such as steering and throttle. While pure Reinforcement Learning (RL) approaches are based exclusively on engineered rewards, Generative Adversarial Imitation Learning (GAIL) agents learn from expert demonstrations while interacting with the environment, which favors GAIL on tasks for which a reward signal is difficult to derive, such as autonomous driving. However, training deep networks directly from raw images on RL tasks is known to be unstable and troublesome. To deal with that, this work proposes a hierarchical GAIL-based architecture (hGAIL) which decouples representation learning from the driving task to solve the autonomous navigation of a vehicle. The proposed architecture consists of two modules: a GAN (Generative Adversarial Net) which generates an abstract mid-level input representation, which is the Bird's-Eye View (BEV) from the surroundings of the vehicle; and the GAIL which learns to control the vehicle based on the BEV predictions from the GAN as input. hGAIL is able to learn both the policy and the mid-level representation simultaneously as the agent interacts with the environment. Our experiments made in the CARLA simulation environment have shown that GAIL exclusively from cameras (without BEV) fails to even learn the task, while hGAIL, after training exclusively on one city, was able to autonomously navigate successfully in 98% of the intersections of a new city not used in training phase. Videos and code available at: https://sites.google.com/view/hgail