# NegoCollab: A Common Representation Negotiation Approach for Heterogeneous Collaborative Perception

arXiv: https://arxiv.org/abs/2510.27647 · HTML: https://mlsanity.com/paper/2510.27647/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 52 references that are in the map.
- First posted on arXiv 31 Oct 2025.

## Builds on (most cited first, 30 of 40)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation](https://mlsanity.com/paper/2205.13542/) (2022)
- [OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communication](https://mlsanity.com/paper/2109.07644/) (2021)
- [V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction](https://mlsanity.com/paper/2008.07519/) (2020)
- [DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detection](https://mlsanity.com/paper/2204.05575/) (2022)
- [Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Maps](https://mlsanity.com/paper/2209.12836/) (2022)
- [CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformers](https://mlsanity.com/paper/2207.02202/) (2022)
- [Swapping Autoencoder for Deep Image Manipulation](https://mlsanity.com/paper/2007.00653/) (2020)
- [Learning Distilled Collaboration Graph for Multi-Agent Perception](https://mlsanity.com/paper/2111.00643/) (2021)
- [F-Cooper: Feature based Cooperative Perception for Autonomous Vehicle Edge Computing System Using 3D Point Clouds](https://mlsanity.com/paper/1909.06459/) (2019)
- [V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perception](https://mlsanity.com/paper/2303.07601/) (2023)
- [Robust Collaborative 3D Object Detection in Presence of Pose Errors](https://mlsanity.com/paper/2211.07214/) (2022)
- [Editable Scene Simulation for Autonomous Driving via Collaborative LLM-Agents](https://mlsanity.com/paper/2402.05746/) (2024)
- [An Extensible Framework for Open Heterogeneous Collaborative Perception](https://mlsanity.com/paper/2401.13964/) (2024)
- [Latency-Aware Collaborative Perception](https://mlsanity.com/paper/2207.08560/) (2022)
- [Collaboration Helps Camera Overtake LiDAR in 3D Detection](https://mlsanity.com/paper/2303.13560/) (2023)
- [HM-ViT: Hetero-modal Vehicle-to-Vehicle Cooperative perception with vision transformer](https://mlsanity.com/paper/2304.10628/) (2023)
- [Communication-Efficient Collaborative Perception via Information Filling with Codebook](https://mlsanity.com/paper/2405.04966/) (2024)
- [BEVDistill: Cross-Modal BEV Distillation for Multi-View 3D Object Detection](https://mlsanity.com/paper/2211.09386/) (2022)
- [Bridging the Domain Gap for Multi-Agent Perception](https://mlsanity.com/paper/2210.08451/) (2022)
- [Asynchrony-Robust Collaborative Perception via Bird's Eye View Flow](https://mlsanity.com/paper/2309.16940/) (2023)
- [PiMAE: Point Cloud and Image Interactive Masked Autoencoders for 3D Object Detection](https://mlsanity.com/paper/2303.08129/) (2023)
- [Model-Agnostic Multi-Agent Perception Framework](https://mlsanity.com/paper/2203.13168/) (2022)
- [STAMP: Scalable Task And Model-agnostic Collaborative Perception](https://mlsanity.com/paper/2501.18616/) (2025)
- [UniDistill: A Universal Cross-Modality Knowledge Distillation Framework for 3D Object Detection in Bird's-Eye View](https://mlsanity.com/paper/2303.15083/) (2023)
- [Among Us: Adversarially Robust Collaborative Perception by Consensus](https://mlsanity.com/paper/2303.09495/) (2023)

## Built on by (most cited first, 5 of 5)

- [BOLT: Online Lightweight Adaptation for Preparation-Free Heterogeneous Cooperative Perception](https://mlsanity.com/paper/2605.00405/) (2026)
- [One Model to Translate Them All: Universal Any-to-Any Translation for Heterogeneous Collaborative Perception](https://mlsanity.com/paper/2605.17907/) (2026)
- [HeteroPROMPT: A Real-time and Privacy-Preserving Heterogeneous Collaborative Perception Framework](https://mlsanity.com/paper/2607.26283/) (2026)
- [CauseCollab: Causal Unified and Modality-Agnostic Network for Heterogeneous Collaborative Perception](https://mlsanity.com/paper/2609.03818/) (2026)
- [PEARL: A Lightweight Prompt-based Feature Interpreter Framework for Real-Time, Anonymous, and Heterogeneous Collaborative Perception](https://mlsanity.com/paper/2609.27123/) (2026)

## Abstract (arXiv)

Collaborative perception improves task performance by expanding the perception range through information sharing among agents. . Immutable heterogeneity poses a significant challenge in collaborative perception, as participating agents may employ different and fixed perception models. This leads to domain gaps in the intermediate features shared among agents, consequently degrading collaborative performance. Aligning the features of all agents to a common representation can eliminate domain gaps with low training cost. However, in existing methods, the common representation is designated as the representation of a specific agent, making it difficult for agents with significant domain discrepancies from this specific agent to achieve proper alignment. This paper proposes NegoCollab, a heterogeneous collaboration method based on the negotiated common representation. It introduces a negotiator during training to derive the common representation from the local representations of each modality's agent, effectively reducing the inherent domain gap with the various local representations. In NegoCollab, the mutual transformation of features between the local representation space and the common representation space is achieved by a pair of sender and receiver. To better align local representations to the common representation containing multimodal information, we introduce structural alignment loss and pragmatic alignment loss in addition to the distribution alignment loss to supervise the training. This enables the knowledge in the common representation to be fully distilled into the sender.