# Communication-Efficient Collaborative Perception via Information Filling with Codebook

arXiv: https://arxiv.org/abs/2405.04966 · HTML: https://mlsanity.com/paper/2405.04966/

## Key facts

- Cited by 68 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 48 references that are in the map.
- First posted on arXiv 8 May 2024.

## Builds on (most cited first, 30 of 35)

- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Categorical Depth Distribution Network for Monocular 3D Object Detection](https://mlsanity.com/paper/2103.01100/) (2021)
- [V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer](https://mlsanity.com/paper/2203.10638/) (2022)
- [OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communication](https://mlsanity.com/paper/2109.07644/) (2021)
- [V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction](https://mlsanity.com/paper/2008.07519/) (2020)
- [DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detection](https://mlsanity.com/paper/2204.05575/) (2022)
- [Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Maps](https://mlsanity.com/paper/2209.12836/) (2022)
- [CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformers](https://mlsanity.com/paper/2207.02202/) (2022)
- [Learning Distilled Collaboration Graph for Multi-Agent Perception](https://mlsanity.com/paper/2111.00643/) (2021)
- [F-Cooper: Feature based Cooperative Perception for Autonomous Vehicle Edge Computing System Using 3D Point Clouds](https://mlsanity.com/paper/1909.06459/) (2019)
- [When2com: Multi-Agent Perception via Communication Graph Grouping](https://mlsanity.com/paper/2006.00176/) (2020)
- [V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perception](https://mlsanity.com/paper/2303.07601/) (2023)
- [Robust Collaborative 3D Object Detection in Presence of Pose Errors](https://mlsanity.com/paper/2211.07214/) (2022)
- [Who2com: Collaborative Perception via Learnable Handshake Communication](https://mlsanity.com/paper/2003.09575/) (2020)
- [V2X-Seq: A Large-Scale Sequential Dataset for Vehicle-Infrastructure Cooperative Perception and Forecasting](https://mlsanity.com/paper/2305.05938/) (2023)
- [An Extensible Framework for Open Heterogeneous Collaborative Perception](https://mlsanity.com/paper/2401.13964/) (2024)
- [Latency-Aware Collaborative Perception](https://mlsanity.com/paper/2207.08560/) (2022)
- [Collaboration Helps Camera Overtake LiDAR in 3D Detection](https://mlsanity.com/paper/2303.13560/) (2023)
- [HM-ViT: Hetero-modal Vehicle-to-Vehicle Cooperative perception with vision transformer](https://mlsanity.com/paper/2304.10628/) (2023)
- [3D Point Cloud Processing and Learning for Autonomous Driving](https://mlsanity.com/paper/2003.00601/) (2020)
- [Bridging the Domain Gap for Multi-Agent Perception](https://mlsanity.com/paper/2210.08451/) (2022)
- [Asynchrony-Robust Collaborative Perception via Bird's Eye View Flow](https://mlsanity.com/paper/2309.16940/) (2023)
- [Spatio-Temporal Domain Awareness for Multi-Agent Collaborative Perception](https://mlsanity.com/paper/2307.13929/) (2023)
- [Learning to Communicate and Correct Pose Errors](https://mlsanity.com/paper/2011.05289/) (2020)
- [Uncertainty Quantification of Collaborative Detection for Self-Driving](https://mlsanity.com/paper/2209.08162/) (2022)
- [End-to-end Learning of Compressible Features](https://mlsanity.com/paper/2007.11797/) (2020)
- [CORE: Cooperative Reconstruction for Multi-Agent Perception](https://mlsanity.com/paper/2307.11514/) (2023)

## Built on by (most cited first, 30 of 68)

- [CoSDH: Communication-Efficient Collaborative Perception via Supply-Demand Awareness and Intermediate-Late Hybridization](https://mlsanity.com/paper/2503.03430/) (2025)
- [V2X-R: Cooperative LiDAR-4D Radar Fusion with Denoising Diffusion for 3D Object Detection](https://mlsanity.com/paper/2411.08402/) (2024)
- [One is Plenty: A Polymorphic Feature Interpreter for Immutable Heterogeneous Collaborative Perception](https://mlsanity.com/paper/2411.16799/) (2024)
- [DSRC: Learning Density-insensitive and Semantic-aware Collaborative Representation against Corruptions](https://mlsanity.com/paper/2412.10739/) (2024)
- [CoCMT: Communication-Efficient Cross-Modal Transformer for Collaborative Perception](https://mlsanity.com/paper/2503.13504/) (2025)
- [CoopTrack: Exploring End-to-End Learning for Efficient Cooperative Sequential Perception](https://mlsanity.com/paper/2507.19239/) (2025)
- [QuantV2X: A Fully Quantized Multi-Agent System for Cooperative Perception](https://mlsanity.com/paper/2509.03704/) (2025)
- [Adver-City: Open-Source Multi-Modal Dataset for Collaborative Perception Under Adverse Weather Conditions](https://mlsanity.com/paper/2410.06380/) (2024)
- [SenseRAG: Constructing Environmental Knowledge Bases with Proactive Querying for LLM-Based Autonomous Driving](https://mlsanity.com/paper/2501.03535/) (2025)
- [Pragmatic Heterogeneous Collaborative Perception via Generative Communication Mechanism](https://mlsanity.com/paper/2510.19618/) (2025)
- [CoPEFT: Fast Adaptation Framework for Multi-Agent Collaborative Perception with Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2502.10705/) (2025)
- [Learning to Detect Objects from Multi-Agent LiDAR Scans without Manual Labels](https://mlsanity.com/paper/2503.08421/) (2025)
- [A Survey of Multi-sensor Fusion Perception for Embodied AI: Background, Methods, Challenges and Prospects](https://mlsanity.com/paper/2506.19769/) (2025)
- [INSTINCT: Instance-Level Interaction Architecture for Query-Based Collaborative Perception](https://mlsanity.com/paper/2509.23700/) (2025)
- [NegoCollab: A Common Representation Negotiation Approach for Heterogeneous Collaborative Perception](https://mlsanity.com/paper/2510.27647/) (2025)
- [Spatial Retrieval Augmented Autonomous Driving](https://mlsanity.com/paper/2512.06865/) (2025)
- [RS-DFM: A Remote Sensing Distributed Foundation Model for Diverse Downstream Tasks](https://mlsanity.com/paper/2406.07032/) (2024)
- [Learning 3D Perception from Others' Predictions](https://mlsanity.com/paper/2410.02646/) (2024)
- [Which2comm: An Efficient Collaborative Perception Framework for 3D Object Detection](https://mlsanity.com/paper/2503.17175/) (2025)
- [SparseCoop: Cooperative Perception with Kinematic-Grounded Queries](https://mlsanity.com/paper/2512.06838/) (2025)
- [CoRA: A Collaborative Robust Architecture with Hybrid Fusion for Efficient Perception](https://mlsanity.com/paper/2512.13191/) (2025)
- [EIMC: Efficient Instance-aware Multi-modal Collaborative Perception](https://mlsanity.com/paper/2603.02532/) (2026)
- [Residual Vector Quantization For Communication-Efficient Multi-Agent Perception](https://mlsanity.com/paper/2509.21464/) (2025)
- [InfoCom: Kilobyte-Scale Communication-Efficient Collaborative Perception with Information Bottleneck](https://mlsanity.com/paper/2512.10305/) (2025)
- [JigsawComm: Joint Semantic Feature Encoding and Transmission for Communication-Efficient Cooperative Perception](https://mlsanity.com/paper/2511.17843/) (2025)
- [FocalComm: Hard Instance-Aware Multi-Agent Perception](https://mlsanity.com/paper/2512.13982/) (2025)
- [Linking Modality Isolation in Heterogeneous Collaborative Perception](https://mlsanity.com/paper/2603.00609/) (2026)
- [RC-GeoCP: Geometric Consensus for 4D Radar-Camera Collaborative Perception](https://mlsanity.com/paper/2603.00654/) (2026)
- [WhisperNet: A Scalable Solution for Bandwidth-Efficient Collaboration](https://mlsanity.com/paper/2603.01708/) (2026)
- [Faster-HEAL: An Efficient and Privacy-Preserving Collaborative Perception Framework for Heterogeneous Autonomous Vehicles](https://mlsanity.com/paper/2603.07314/) (2026)

## Abstract (arXiv)

Collaborative perception empowers each agent to improve its perceptual ability through the exchange of perceptual messages with other agents. It inherently results in a fundamental trade-off between perception ability and communication cost. To address this bottleneck issue, our core idea is to optimize the collaborative messages from two key aspects: representation and selection. The proposed codebook-based message representation enables the transmission of integer codes, rather than high-dimensional feature maps. The proposed information-filling-driven message selection optimizes local messages to collectively fill each agent's information demand, preventing information overflow among multiple agents. By integrating these two designs, we propose CodeFilling, a novel communication-efficient collaborative perception system, which significantly advances the perception-communication trade-off and is inclusive to both homogeneous and heterogeneous collaboration settings. We evaluate CodeFilling in both a real-world dataset, DAIR-V2X, and a new simulation dataset, OPV2VH+. Results show that CodeFilling outperforms previous SOTA Where2comm on DAIR-V2X/OPV2VH+ with 1,333/1,206 times lower communication volume. Our code is available at https://github.com/PhyllisH/CodeFilling.