# Collaborative Perception for Connected and Autonomous Driving: Challenges, Possible Solutions and Opportunities

arXiv: https://arxiv.org/abs/2401.01544 · HTML: https://mlsanity.com/paper/2401.01544/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 22 references that are in the map.
- First posted on arXiv 3 Jan 2024.

## Builds on (most cited first, 16 of 16)

- [V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer](https://mlsanity.com/paper/2203.10638/) (2022)
- [OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communication](https://mlsanity.com/paper/2109.07644/) (2021)
- [V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction](https://mlsanity.com/paper/2008.07519/) (2020)
- [Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Maps](https://mlsanity.com/paper/2209.12836/) (2022)
- [CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformers](https://mlsanity.com/paper/2207.02202/) (2022)
- [Learning Distilled Collaboration Graph for Multi-Agent Perception](https://mlsanity.com/paper/2111.00643/) (2021)
- [F-Cooper: Feature based Cooperative Perception for Autonomous Vehicle Edge Computing System Using 3D Point Clouds](https://mlsanity.com/paper/1909.06459/) (2019)
- [V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Driving](https://mlsanity.com/paper/2202.08449/) (2022)
- [V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perception](https://mlsanity.com/paper/2303.07601/) (2023)
- [Robust Collaborative 3D Object Detection in Presence of Pose Errors](https://mlsanity.com/paper/2211.07214/) (2022)
- [Collaborative Perception in Autonomous Driving: Methods, Datasets and Challenges](https://mlsanity.com/paper/2301.06262/) (2023)
- [Latency-Aware Collaborative Perception](https://mlsanity.com/paper/2207.08560/) (2022)
- [Adaptive Communications in Collaborative Perception with Domain Alignment for Autonomous Driving](https://mlsanity.com/paper/2310.00013/) (2023)
- [Vehicle as a Service (VaaS): Leverage Vehicles to Build Service Networks and Capabilities for Smart Cities](https://mlsanity.com/paper/2304.11397/) (2023)
- [Towards Full-scene Domain Generalization in Multi-agent Collaborative Bird's Eye View Segmentation for Connected and Autonomous Driving](https://mlsanity.com/paper/2311.16754/) (2023)
- [Malicious Agent Detection for Robust Multi-Agent Collaborative Perception](https://mlsanity.com/paper/2310.11901/) (2023)

## Built on by (most cited first, 30 of 32)

- [Mobile Edge Intelligence for Large Language Models: A Contemporary Survey](https://mlsanity.com/paper/2407.18921/) (2024)
- [AdaptSFL: Adaptive Split Federated Learning in Resource-constrained Edge Networks](https://mlsanity.com/paper/2403.13101/) (2024)
- [SplitLoRA: A Split Parameter-Efficient Fine-Tuning Framework for Large Language Models](https://mlsanity.com/paper/2407.00952/) (2024)
- [Automated Federated Pipeline for Parameter-Efficient Fine-Tuning of Large Language Models](https://mlsanity.com/paper/2404.06448/) (2024)
- [STAMP: Scalable Task And Model-agnostic Collaborative Perception](https://mlsanity.com/paper/2501.18616/) (2025)
- [AgentsCoDriver: Large Language Model Empowered Collaborative Driving with Lifelong Learning](https://mlsanity.com/paper/2404.06345/) (2024)
- [FedAC: An Adaptive Clustered Federated Learning Framework for Heterogeneous Data](https://mlsanity.com/paper/2403.16460/) (2024)
- [AgentsCoMerge: Large Language Model Empowered Collaborative Decision Making for Ramp Merging](https://mlsanity.com/paper/2408.03624/) (2024)
- [SatFed: A Resource-Efficient LEO Satellite-Assisted Heterogeneous Federated Learning Framework](https://mlsanity.com/paper/2409.13503/) (2024)
- [Multi-Agent Autonomous Driving Systems with Large Language Models: A Survey of Recent Advances](https://mlsanity.com/paper/2502.16804/) (2025)
- [IC3M: In-Car Multimodal Multi-object Monitoring for Abnormal Status of Both Driver and Passengers](https://mlsanity.com/paper/2410.02592/) (2024)
- [Directed-CP: Directed Collaborative Perception for Connected and Autonomous Vehicles via Proactive Attention](https://mlsanity.com/paper/2409.08840/) (2024)
- [CP-Guard: Malicious Agent Detection and Defense in Collaborative Bird's Eye View Perception](https://mlsanity.com/paper/2412.12000/) (2024)
- [SmartCooper: Vehicular Collaborative Perception with Adaptive Fusion and Judger Mechanism](https://mlsanity.com/paper/2402.00321/) (2024)
- [Dynamic Uncertainty-aware Multimodal Fusion for Outdoor Health Monitoring](https://mlsanity.com/paper/2508.09085/) (2025)
- [GCP: Guarded Collaborative Perception with Spatial-Temporal Aware Malicious Agent Detection](https://mlsanity.com/paper/2501.02450/) (2025)
- [Generative AI for Autonomous Driving: Frontiers and Opportunities](https://mlsanity.com/paper/2505.08854/) (2025)
- [Distribution-Aligned Decoding for Efficient LLM Task Adaptation](https://mlsanity.com/paper/2509.15888/) (2025)
- [Automated Vehicles Should be Connected with Natural Language](https://mlsanity.com/paper/2507.01059/) (2025)
- [PartialLoading: User Scheduling and Bandwidth Allocation for Parameter-sharing Edge Inference](https://mlsanity.com/paper/2503.22982/) (2025)
- [Learning Mutual View Information Graph for Adaptive Adversarial Collaborative Perception](https://mlsanity.com/paper/2602.19596/) (2026)
- [Channel-Aware Throughput Maximization for Cooperative Data Fusion in CAV](https://mlsanity.com/paper/2410.04320/) (2024)
- [Task-Aware Parameter-Efficient Fine-Tuning of Large Pre-Trained Models at the Edge](https://mlsanity.com/paper/2504.03718/) (2025)
- [SafeCoop: Unravelling Full Stack Safety in Agentic Collaborative Driving](https://mlsanity.com/paper/2510.18123/) (2025)
- [CP-uniGuard: A Unified, Probability-Agnostic, and Adaptive Framework for Malicious Agent Detection and Defense in Multi-Agent Embodied Perception Systems](https://mlsanity.com/paper/2506.22890/) (2025)
- [V2VCrafter: Consistent Street-View Image Generation Across Vehicles](https://mlsanity.com/paper/2605.29471/) (2026)
- [Learning from Distributed Eyes: Leveraging Collaborative Perception for Automated Model Adaptation](https://mlsanity.com/paper/2609.18511/) (2026)
- [When Autonomous Vehicle Meets V2X Cooperative Perception: How Far Are We?](https://mlsanity.com/paper/2509.24927/) (2025)
- [DSP-Reg: Domain-Sensitive Parameter Regularization for Robust Domain Generalization](https://mlsanity.com/paper/2601.19394/) (2026)
- [One Model to Translate Them All: Universal Any-to-Any Translation for Heterogeneous Collaborative Perception](https://mlsanity.com/paper/2605.17907/) (2026)

## Abstract (arXiv)

Autonomous driving has attracted significant attention from both academia and industries, which is expected to offer a safer and more efficient driving system. However, current autonomous driving systems are mostly based on a single vehicle, which has significant limitations which still poses threats to driving safety. Collaborative perception with connected and autonomous vehicles (CAVs) shows a promising solution to overcoming these limitations. In this article, we first identify the challenges of collaborative perception, such as data sharing asynchrony, data volume, and pose errors. Then, we discuss the possible solutions to address these challenges with various technologies, where the research opportunities are also elaborated. Furthermore, we propose a scheme to deal with communication efficiency and latency problems, which is a channel-aware collaborative perception framework to dynamically adjust the communication graph and minimize latency, thereby improving perception performance while increasing communication efficiency. Finally, we conduct experiments to demonstrate the effectiveness of our proposed scheme.