# Learning Distilled Collaboration Graph for Multi-Agent Perception

arXiv: https://arxiv.org/abs/2111.00643 · HTML: https://mlsanity.com/paper/2111.00643/

## Key facts

- Cited by 194 papers in the map (data through 14 Oct 2026).
- Placed in General knowledge distillation methods, one of 12 directions in Knowledge distillation.
- Builds on 30 of its 39 references that are in the map.
- First posted on arXiv 1 Nov 2021.

## Placed in

- Knowledge distillation / [General knowledge distillation methods](https://mlsanity.com/distill/general-knowledge-distillation-methods/)

## Builds on (most cited first, 30 of 30)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [Image Segmentation Using Deep Learning: A Survey](https://mlsanity.com/paper/2001.05566/) (2020)
- [Learning to Communicate with Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1605.06676/) (2016)
- [Deep Learning for Generic Object Detection: A Survey](https://mlsanity.com/paper/1809.02165/) (2018)
- [Large-scale Point Cloud Semantic Segmentation with Superpoint Graphs](https://mlsanity.com/paper/1711.09869/) (2017)
- [From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network](https://mlsanity.com/paper/1907.03670/) (2019)
- [Learning Multiagent Communication with Backpropagation](https://mlsanity.com/paper/1605.07736/) (2016)
- [Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud](https://mlsanity.com/paper/2003.01251/) (2020)
- [V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction](https://mlsanity.com/paper/2008.07519/) (2020)
- [Structured Knowledge Distillation for Dense Prediction](https://mlsanity.com/paper/1903.04197/) (2019)
- [Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net](https://mlsanity.com/paper/2012.12395/) (2020)
- [TarMAC: Targeted Multi-Agent Communication](https://mlsanity.com/paper/1810.11187/) (2018)
- [Learning Attentional Communication for Multi-Agent Cooperation](https://mlsanity.com/paper/1805.07733/) (2018)
- [Cooper: Cooperative Perception for Connected Autonomous Vehicles based on 3D Point Clouds](https://mlsanity.com/paper/1905.05265/) (2019)
- [When2com: Multi-Agent Perception via Communication Graph Grouping](https://mlsanity.com/paper/2006.00176/) (2020)
- [Large scale distributed neural network training through online distillation](https://mlsanity.com/paper/1804.03235/) (2018)
- [Who2com: Collaborative Perception via Learnable Handshake Communication](https://mlsanity.com/paper/2003.09575/) (2020)
- [Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks](https://mlsanity.com/paper/1812.09755/) (2018)
- [3D Point Cloud Processing and Learning for Autonomous Driving](https://mlsanity.com/paper/2003.00601/) (2020)
- [Deep Learning for Visual Tracking: A Comprehensive Survey](https://mlsanity.com/paper/1912.00535/) (2019)
- [VAIN: Attentional Multi-agent Predictive Modeling](https://mlsanity.com/paper/1706.06122/) (2017)
- [Learning to Communicate and Correct Pose Errors](https://mlsanity.com/paper/2011.05289/) (2020)
- [Uncertainty-Aware Multi-Shot Knowledge Distillation for Image-Based Object Re-Identification](https://mlsanity.com/paper/2001.05197/) (2020)
- [Fooling LiDAR Perception via Adversarial Trajectory Perturbation](https://mlsanity.com/paper/2103.15326/) (2021)
- [Multi-Frame to Single-Frame: Knowledge Distillation for 3D Object Detection](https://mlsanity.com/paper/2009.11859/) (2020)

## Built on by (most cited first, 30 of 194)

- [V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer](https://mlsanity.com/paper/2203.10638/) (2022)
- [VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completion](https://mlsanity.com/paper/2302.12251/) (2023)
- [DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detection](https://mlsanity.com/paper/2204.05575/) (2022)
- [Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Maps](https://mlsanity.com/paper/2209.12836/) (2022)
- [CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformers](https://mlsanity.com/paper/2207.02202/) (2022)
- [Large Language Model Agent: A Survey on Methodology, Applications and Challenges](https://mlsanity.com/paper/2503.21460/) (2025)
- [3D Object Detection for Autonomous Driving: A Comprehensive Survey](https://mlsanity.com/paper/2206.09474/) (2022)
- [Robust Collaborative 3D Object Detection in Presence of Pose Errors](https://mlsanity.com/paper/2211.07214/) (2022)
- [V2X-Seq: A Large-Scale Sequential Dataset for Vehicle-Infrastructure Cooperative Perception and Forecasting](https://mlsanity.com/paper/2305.05938/) (2023)
- [COOPERNAUT: End-to-End Driving with Cooperative Perception for Networked Vehicles](https://mlsanity.com/paper/2205.02222/) (2022)
- [Collaborative Perception in Autonomous Driving: Methods, Datasets and Challenges](https://mlsanity.com/paper/2301.06262/) (2023)
- [An Extensible Framework for Open Heterogeneous Collaborative Perception](https://mlsanity.com/paper/2401.13964/) (2024)
- [Latency-Aware Collaborative Perception](https://mlsanity.com/paper/2207.08560/) (2022)
- [Collaboration Helps Camera Overtake LiDAR in 3D Detection](https://mlsanity.com/paper/2303.13560/) (2023)
- [SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Driving](https://mlsanity.com/paper/2306.09001/) (2023)
- [DeepAccident: A Motion and Accident Prediction Benchmark for V2X Autonomous Driving](https://mlsanity.com/paper/2304.01168/) (2023)
- [HM-ViT: Hetero-modal Vehicle-to-Vehicle Cooperative perception with vision transformer](https://mlsanity.com/paper/2304.10628/) (2023)
- [Communication-Efficient Collaborative Perception via Information Filling with Codebook](https://mlsanity.com/paper/2405.04966/) (2024)
- [Bridging the Domain Gap for Multi-Agent Perception](https://mlsanity.com/paper/2210.08451/) (2022)
- [Asynchrony-Robust Collaborative Perception via Bird's Eye View Flow](https://mlsanity.com/paper/2309.16940/) (2023)
- [Spatio-Temporal Domain Awareness for Multi-Agent Collaborative Perception](https://mlsanity.com/paper/2307.13929/) (2023)
- [Uncertainty Quantification of Collaborative Detection for Self-Driving](https://mlsanity.com/paper/2209.08162/) (2022)
- [STAMP: Scalable Task And Model-agnostic Collaborative Perception](https://mlsanity.com/paper/2501.18616/) (2025)
- [End-to-End Autonomous Driving through V2X Cooperation](https://mlsanity.com/paper/2404.00717/) (2024)
- [CORE: Cooperative Reconstruction for Multi-Agent Perception](https://mlsanity.com/paper/2307.11514/) (2023)
- [Towards Vehicle-to-everything Autonomous Driving: A Survey on Collaborative Perception](https://mlsanity.com/paper/2308.16714/) (2023)
- [RCooper: A Real-world Large-scale Dataset for Roadside Cooperative Perception](https://mlsanity.com/paper/2403.10145/) (2024)
- [Among Us: Adversarially Robust Collaborative Perception by Consensus](https://mlsanity.com/paper/2303.09495/) (2023)
- [DI-V2X: Learning Domain-Invariant Representation for Vehicle-Infrastructure Collaborative 3D Object Detection](https://mlsanity.com/paper/2312.15742/) (2023)
- [Collaborative Semantic Occupancy Prediction with Hybrid Feature Fusion in Connected Automated Vehicles](https://mlsanity.com/paper/2402.07635/) (2024)

## Abstract (arXiv)

To promote better performance-bandwidth trade-off for multi-agent perception, we propose a novel distilled collaboration graph (DiscoGraph) to model trainable, pose-aware, and adaptive collaboration among agents. Our key novelties lie in two aspects. First, we propose a teacher-student framework to train DiscoGraph via knowledge distillation. The teacher model employs an early collaboration with holistic-view inputs; the student model is based on intermediate collaboration with single-view inputs. Our framework trains DiscoGraph by constraining post-collaboration feature maps in the student model to match the correspondences in the teacher model. Second, we propose a matrix-valued edge weight in DiscoGraph. In such a matrix, each element reflects the inter-agent attention at a specific spatial region, allowing an agent to adaptively highlight the informative regions. During inference, we only need to use the student model named as the distilled collaboration network (DiscoNet). Attributed to the teacher-student framework, multiple agents with the shared DiscoNet could collaboratively approach the performance of a hypothetical teacher model with a holistic view. Our approach is validated on V2X-Sim 1.0, a large-scale multi-agent perception dataset that we synthesized using CARLA and SUMO co-simulation. Our quantitative and qualitative experiments in multi-agent 3D object detection show that DiscoNet could not only achieve a better performance-bandwidth trade-off than the state-of-the-art collaborative perception methods, but also bring more straightforward design rationale. Our code is available on https://github.com/ai4ce/DiscoNet.