# CoCMT: Communication-Efficient Cross-Modal Transformer for Collaborative Perception

arXiv: https://arxiv.org/abs/2503.13504 · HTML: https://mlsanity.com/paper/2503.13504/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 55 references that are in the map.
- First posted on arXiv 13 Mar 2025.

## Builds on (most cited first, 30 of 43)

- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation](https://mlsanity.com/paper/2205.13542/) (2022)
- [PETR: Position Embedding Transformation for Multi-View 3D Object Detection](https://mlsanity.com/paper/2203.05625/) (2022)
- [DN-DETR: Accelerate DETR Training by Introducing Query DeNoising](https://mlsanity.com/paper/2203.01305/) (2022)
- [V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer](https://mlsanity.com/paper/2203.10638/) (2022)
- [Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection](https://mlsanity.com/paper/2303.11926/) (2023)
- [OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communication](https://mlsanity.com/paper/2109.07644/) (2021)
- [V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction](https://mlsanity.com/paper/2008.07519/) (2020)
- [Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Maps](https://mlsanity.com/paper/2209.12836/) (2022)
- [CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformers](https://mlsanity.com/paper/2207.02202/) (2022)
- [Learning Distilled Collaboration Graph for Multi-Agent Perception](https://mlsanity.com/paper/2111.00643/) (2021)
- [Cooper: Cooperative Perception for Connected Autonomous Vehicles based on 3D Point Clouds](https://mlsanity.com/paper/1905.05265/) (2019)
- [Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusion](https://mlsanity.com/paper/2211.10581/) (2022)
- [V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perception](https://mlsanity.com/paper/2303.07601/) (2023)
- [Cooperative Perception for 3D Object Detection in Driving Scenarios using Infrastructure Sensors](https://mlsanity.com/paper/1912.12147/) (2019)
- [Sparse4D v2: Recurrent Temporal Fusion with Sparse Model](https://mlsanity.com/paper/2305.14018/) (2023)
- [OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Driving](https://mlsanity.com/paper/2412.15208/) (2024)
- [An Extensible Framework for Open Heterogeneous Collaborative Perception](https://mlsanity.com/paper/2401.13964/) (2024)
- [Cross Modal Transformer: Towards Fast and Robust 3D Object Detection](https://mlsanity.com/paper/2301.01283/) (2023)
- [HM-ViT: Hetero-modal Vehicle-to-Vehicle Cooperative perception with vision transformer](https://mlsanity.com/paper/2304.10628/) (2023)
- [Sparse4D v3: Advancing End-to-End 3D Detection and Tracking](https://mlsanity.com/paper/2311.11722/) (2023)
- [Communication-Efficient Collaborative Perception via Information Filling with Codebook](https://mlsanity.com/paper/2405.04966/) (2024)
- [DSDNet: Deep Structured self-Driving Network](https://mlsanity.com/paper/2008.06041/) (2020)
- [Asynchrony-Robust Collaborative Perception via Bird's Eye View Flow](https://mlsanity.com/paper/2309.16940/) (2023)
- [Uncertainty Quantification of Collaborative Detection for Self-Driving](https://mlsanity.com/paper/2209.08162/) (2022)
- [STAMP: Scalable Task And Model-agnostic Collaborative Perception](https://mlsanity.com/paper/2501.18616/) (2025)
- [CORE: Cooperative Reconstruction for Multi-Agent Perception](https://mlsanity.com/paper/2307.11514/) (2023)
- [Towards Vehicle-to-everything Autonomous Driving: A Survey on Collaborative Perception](https://mlsanity.com/paper/2308.16714/) (2023)

## Built on by (most cited first, 13 of 13)

- [LangCoop: Collaborative Driving with Language](https://mlsanity.com/paper/2504.13406/) (2025)
- [V2X-UniPool: Unifying Multimodal Perception and Knowledge Reasoning for Autonomous Driving](https://mlsanity.com/paper/2506.02580/) (2025)
- [AirV2X: Unified Air-Ground Vehicle-to-Everything Collaboration](https://mlsanity.com/paper/2506.19283/) (2025)
- [Automated Vehicles Should be Connected with Natural Language](https://mlsanity.com/paper/2507.01059/) (2025)
- [Digital Twin AI: Opportunities and Challenges from Large Language Models to World Models](https://mlsanity.com/paper/2601.01321/) (2026)
- [SafeCoop: Unravelling Full Stack Safety in Agentic Collaborative Driving](https://mlsanity.com/paper/2510.18123/) (2025)
- [Faster-HEAL: An Efficient and Privacy-Preserving Collaborative Perception Framework for Heterogeneous Autonomous Vehicles](https://mlsanity.com/paper/2603.07314/) (2026)
- [Is Intermediate Fusion All You Need for UAV-based Collaborative Perception?](https://mlsanity.com/paper/2504.21774/) (2025)
- [Dense Coverage, Sparse Refinement: Byte-Constrained Cooperative Perception](https://mlsanity.com/paper/2609.29456/) (2026)
- [WaveComm: Lightweight Communication for Collaborative Perception via Wavelet Feature Distillation](https://mlsanity.com/paper/2603.13365/) (2026)
- [UECP: Uncertainty-Enhanced Collaborative Perception](https://mlsanity.com/paper/2606.23046/) (2026)
- [HydraCollab: Adaptive Collaborative-Perception for Distributed Autonomous Systems](https://mlsanity.com/paper/2607.00191/) (2026)
- [LR-V2X: Loss-Resilient Collaborative Perception under Low-Bandwidth Communication](https://mlsanity.com/paper/2610.11264/) (2026)

## Abstract (arXiv)

Multi-agent collaborative perception enhances each agent perceptual capabilities by sharing sensing information to cooperatively perform robot perception tasks. This approach has proven effective in addressing challenges such as sensor deficiencies, occlusions, and long-range perception. However, existing representative collaborative perception systems transmit intermediate feature maps, such as bird-eye view (BEV) representations, which contain a significant amount of non-critical information, leading to high communication bandwidth requirements. To enhance communication efficiency while preserving perception capability, we introduce CoCMT, an object-query-based collaboration framework that optimizes communication bandwidth by selectively extracting and transmitting essential features. Within CoCMT, we introduce the Efficient Query Transformer (EQFormer) to effectively fuse multi-agent object queries and implement a synergistic deep supervision to enhance the positive reinforcement between stages, leading to improved overall performance. Experiments on OPV2V and V2V4Real datasets show CoCMT outperforms state-of-the-art methods while drastically reducing communication needs. On V2V4Real, our model (Top-50 object queries) requires only 0.416 Mb bandwidth, 83 times less than SOTA methods, while improving AP70 by 1.1 percent. This efficiency breakthrough enables practical collaborative perception deployment in bandwidth-constrained environments without sacrificing detection accuracy.