# End-to-End Autonomous Driving through V2X Cooperation

arXiv: https://arxiv.org/abs/2404.00717 · HTML: https://mlsanity.com/paper/2404.00717/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Placed in Cooperative and V2X driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 55 of its 67 references that are in the map.
- First posted on arXiv 31 Mar 2024.

## Placed in

- End-to-end autonomous driving / [Cooperative and V2X driving](https://mlsanity.com/driving/cooperative-and-v2x-driving/)

## Builds on (most cited first, 30 of 55)

- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [End-to-end Driving via Conditional Imitation Learning](https://mlsanity.com/paper/1710.02410/) (2017)
- [Soft-NMS -- Improving Object Detection With One Line of Code](https://mlsanity.com/paper/1704.04503/) (2017)
- [Multi-Modal Fusion Transformer for End-to-End Autonomous Driving](https://mlsanity.com/paper/2104.09224/) (2021)
- [Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Driving](https://mlsanity.com/paper/2304.14365/) (2023)
- [V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer](https://mlsanity.com/paper/2203.10638/) (2022)
- [SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving](https://mlsanity.com/paper/2303.09551/) (2023)
- [OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communication](https://mlsanity.com/paper/2109.07644/) (2021)
- [V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction](https://mlsanity.com/paper/2008.07519/) (2020)
- [Learning to Drive in a Day](https://mlsanity.com/paper/1807.00412/) (2018)
- [DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detection](https://mlsanity.com/paper/2204.05575/) (2022)
- [Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Maps](https://mlsanity.com/paper/2209.12836/) (2022)
- [Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baseline](https://mlsanity.com/paper/2206.08129/) (2022)
- [End-to-end Interpretable Neural Motion Planner](https://mlsanity.com/paper/2101.06679/) (2021)
- [Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?](https://mlsanity.com/paper/2312.03031/) (2023)
- [FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Cameras](https://mlsanity.com/paper/2104.10490/) (2021)
- [CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformers](https://mlsanity.com/paper/2207.02202/) (2022)
- [Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer](https://mlsanity.com/paper/2207.14024/) (2022)
- [Learning Distilled Collaboration Graph for Multi-Agent Perception](https://mlsanity.com/paper/2111.00643/) (2021)
- [BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Driving](https://mlsanity.com/paper/2205.09743/) (2022)
- [Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenes](https://mlsanity.com/paper/2305.10430/) (2023)
- [Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving](https://mlsanity.com/paper/2305.06242/) (2023)
- [Cooper: Cooperative Perception for Connected Autonomous Vehicles based on 3D Point Clouds](https://mlsanity.com/paper/1905.05265/) (2019)
- [V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Driving](https://mlsanity.com/paper/2202.08449/) (2022)
- [Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representations](https://mlsanity.com/paper/2008.05930/) (2020)

## Built on by (most cited first, 30 of 42)

- [V2X-VLM: End-to-End V2X Cooperative Autonomous Driving Through Large Vision-Language Models](https://mlsanity.com/paper/2408.09251/) (2024)
- [V2XPnP: Vehicle-to-Everything Spatio-Temporal Fusion for Multi-Agent Perception and Prediction](https://mlsanity.com/paper/2412.01812/) (2024)
- [V2X-Radar: A Multi-modal Dataset with 4D Radar for Cooperative Perception](https://mlsanity.com/paper/2411.10962/) (2024)
- [Challenger: Affordable Adversarial Driving Video Generation](https://mlsanity.com/paper/2505.15880/) (2025)
- [Griffin: Aerial-Ground Cooperative Detection and Tracking Dataset and Benchmark](https://mlsanity.com/paper/2503.06983/) (2025)
- [DSRC: Learning Density-insensitive and Semantic-aware Collaborative Representation against Corruptions](https://mlsanity.com/paper/2412.10739/) (2024)
- [CoopTrack: Exploring End-to-End Learning for Efficient Cooperative Sequential Perception](https://mlsanity.com/paper/2507.19239/) (2025)
- [CoopDETR: A Unified Cooperative Perception Framework for 3D Detection via Object Query](https://mlsanity.com/paper/2502.19313/) (2025)
- [Research Challenges and Progress in the End-to-End V2X Cooperative Autonomous Driving Competition](https://mlsanity.com/paper/2507.21610/) (2025)
- [TurboTrain: Towards Efficient and Balanced Multi-Task Learning for Multi-Agent Perception and Prediction](https://mlsanity.com/paper/2508.04682/) (2025)
- [M3CAD: Towards Generic Cooperative Autonomous Driving Benchmark](https://mlsanity.com/paper/2505.06746/) (2025)
- [Risk Map As Middleware: Towards Interpretable Cooperative End-to-end Autonomous Driving for Risk-Aware Planning](https://mlsanity.com/paper/2508.07686/) (2025)
- [DriveE2E: Closed-Loop Benchmark for End-to-End Autonomous Driving through Real-to-Simulation](https://mlsanity.com/paper/2509.23922/) (2025)
- [MDrive: Benchmarking Closed-Loop Cooperative Driving for End-to-End Multi-agent Systems](https://mlsanity.com/paper/2605.10904/) (2026)
- [LiDAR-based End-to-end Temporal Perception for Vehicle-Infrastructure Cooperation](https://mlsanity.com/paper/2411.14927/) (2024)
- [MIC-BEV: Multi-Infrastructure Camera Bird's-Eye-View Transformer with Relation-Aware Fusion for 3D Object Detection](https://mlsanity.com/paper/2510.24688/) (2025)
- [SparseCoop: Cooperative Perception with Kinematic-Grounded Queries](https://mlsanity.com/paper/2512.06838/) (2025)
- [SEAL: Vision-Language Model-Based Safe End-to-End Cooperative Autonomous Driving with Adaptive Long-Tail Modeling](https://mlsanity.com/paper/2506.21041/) (2025)
- [AsyncBEV: Cross-modal Flow Alignment in Asynchronous 3D Object Detection](https://mlsanity.com/paper/2601.12994/) (2026)
- [OmniV2X: A Generative Foundation Planner for Efficient End-to-End Cooperative Driving](https://mlsanity.com/paper/2606.21165/) (2026)
- [OWMDrive: Causality-Aware End-to-End Autonomous Driving via 4D Occupancy World Model](https://mlsanity.com/paper/2606.30421/) (2026)
- [SpaRC-AD: A Baseline for Radar-Camera Fusion in End-to-End Autonomous Driving](https://mlsanity.com/paper/2508.10567/) (2025)
- [MAP: End-to-End Autonomous Driving with Map-Assisted Planning](https://mlsanity.com/paper/2509.13926/) (2025)
- [From Features to Reference Points: Lightweight and Adaptive Fusion for Cooperative Autonomous Driving](https://mlsanity.com/paper/2511.18757/) (2025)
- [Long-SCOPE: Fully Sparse Long-Range Cooperative 3D Perception](https://mlsanity.com/paper/2604.09206/) (2026)
- [Defer to Plan: Adaptive Multi-Agent Fusion for End-to-End V2X Driving](https://mlsanity.com/paper/2607.19774/) (2026)
- [Wireless Communication as an Information Sensor for Multi-agent Cooperative Perception: A Survey](https://mlsanity.com/paper/2505.00747/) (2025)
- [A Survey on Deep Multi-Task Learning in Connected Autonomous Vehicles](https://mlsanity.com/paper/2508.00917/) (2025)
- [Fault-Tolerant MARL for CAVs under Observation Perturbations for Highway On-Ramp Merging](https://mlsanity.com/paper/2511.23193/) (2025)
- [LADY: Linear Attention for Autonomous Driving Efficiency without Transformers](https://mlsanity.com/paper/2512.15038/) (2025)

## Abstract (arXiv)

Cooperatively utilizing both ego-vehicle and infrastructure sensor data via V2X communication has emerged as a promising approach for advanced autonomous driving. However, current research mainly focuses on improving individual modules, rather than taking end-to-end learning to optimize final planning performance, resulting in underutilized data potential. In this paper, we introduce UniV2X, a pioneering cooperative autonomous driving framework that seamlessly integrates all key driving modules across diverse views into a unified network. We propose a sparse-dense hybrid data transmission and fusion mechanism for effective vehicle-infrastructure cooperation, offering three advantages: 1) Effective for simultaneously enhancing agent perception, online mapping, and occupancy prediction, ultimately improving planning performance. 2) Transmission-friendly for practical and limited communication conditions. 3) Reliable data fusion with interpretability of this hybrid data. We implement UniV2X, as well as reproducing several benchmark methods, on the challenging DAIR-V2X, the real-world cooperative driving dataset. Experimental results demonstrate the effectiveness of UniV2X in significantly enhancing planning performance, as well as all intermediate output performance. The project is available at hrefhttps://github.com/AIR-THU/UniV2Xhttps://github.com/AIR-THU/UniV2X.