Closed-Loop Evaluation of Bird's-Eye-View Maps from Cross-View Transformers as Inputs to Behavior-Cloning Policies
On the map: Offline RL and imitation learning from data → Imitation and offline RL for driving
Abstract (arXiv)
In autonomous driving, Bird's-Eye View (BEV) representations provide a structured, top-down abstraction of the vehicle's surroundings and have become a key input modality for Behavioral Cloning (BC) policies. While ground-truth BEV maps are readily available in simulation, real-world deployment requires replacing them with camera-predicted counterparts - a substitution that introduces perceptual errors whose downstream impact on closed-loop driving performance is not well understood. In this work, we investigate the use of Cross-View Transformer (CVT)-predicted BEV maps as direct policy inputs for a BC agent in the CARLA simulator. We propose a six-channel BEV representation covering road surface, planned route, lane boundaries, vehicles, pedestrians, and traffic lights, and introduce a Kernel Density Estimation (KDE) weighting scheme that rebalances the segmentation loss towards underrepresented driving maneuvers such as curves and intersections. Closed-loop evaluation across two CARLA towns shows that the KDE-weighted model is the only predicted-BEV agent to complete a full episode without infractions, despite not achieving the highest aggregate IoU. This discrepancy reveals that global segmentation metrics are poor proxies for driving performance: what determines navigation success is prediction quality at geometrically critical locations, and the route channel emerges as the primary bottleneck for reliable agent navigation under predicted BEV inputs.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Imitation and offline RL for driving, one of 13 directions in Offline RL and imitation learning from data. Builds on 9 of its 13 references that are in the map. First posted on arXiv 5 Sep 2026.
Builds on
9 of 13 references.
Closest ancestors · 2
Same direction, newest first
- 2023Hierarchical Generative Adversarial Imitation Learning with Mid-level Input Generation for Autonomous Driving on Urban Environments9 Feb
- 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst7 Dec
Other references · 7
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
- 2016End to End Learning for Self-Driving Carscited by 1,009
- 2017End-to-end Driving via Conditional Imitation Learningcited by 485
- 2022Cross-view Transformers for real-time Map-view Semantic Segmentationcited by 264
- 2021Training Larger Networks for Deep Reinforcement Learningcited by 38
- 2021Proximal Policy Optimization with Continuous Bounded Action Space via the Beta Distributioncited by 10
- 2025An Initial Study of Bird's-Eye View Generation for Autonomous Vehicles using Cross-View Transformerscited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Investigating Behavior Cloning from Few Demonstrations for Autonomous Driving based on Bird's-Eye View in Simulated Cities
- 2025Progressive Bird's Eye View Perception for Safety-Critical Autonomous Driving: A Comprehensive Survey
- 2024CarFormer: Self-Driving with Learned Object-Centric Representations
- 2020Driving Through Ghosts: Behavioral Cloning with False Positives
- 2026Diffusion-2BC: Hybrid Diffusion and Regression Training for Offline Behavior Cloning in Autonomous Driving
- 2026Imitation Learning for Autonomous Driving in CARLA
- 2024BEVWorld: A Multimodal World Simulator for Autonomous Driving via Scene-Level BEV Latents
- 2025BEVTraj: Map-Free End-to-End Trajectory Prediction in Bird's-Eye View with Deformable Attention and Sparse Goal Proposals