# Practical Collaborative Perception: A Framework for Asynchronous and Multi-Agent 3D Object Detection

arXiv: https://arxiv.org/abs/2307.01462 · HTML: https://mlsanity.com/paper/2307.01462/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 45 references that are in the map.
- First posted on arXiv 4 Jul 2023.

## Builds on (most cited first, 30 of 36)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection](https://mlsanity.com/paper/1908.09492/) (2019)
- [V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer](https://mlsanity.com/paper/2203.10638/) (2022)
- [V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction](https://mlsanity.com/paper/2008.07519/) (2020)
- [DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detection](https://mlsanity.com/paper/2204.05575/) (2022)
- [CoverNet: Multimodal Behavior Prediction using Trajectory Sets](https://mlsanity.com/paper/1911.10298/) (2019)
- [MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Prediction](https://mlsanity.com/paper/2111.14973/) (2021)
- [FlowNet3D: Learning Scene Flow in 3D Point Clouds](https://mlsanity.com/paper/1806.01411/) (2018)
- [Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net](https://mlsanity.com/paper/2012.12395/) (2020)
- [CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformers](https://mlsanity.com/paper/2207.02202/) (2022)
- [Learning Distilled Collaboration Graph for Multi-Agent Perception](https://mlsanity.com/paper/2111.00643/) (2021)
- [V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Driving](https://mlsanity.com/paper/2202.08449/) (2022)
- [FLOT: Scene Flow on Point Clouds Guided by Optimal Transport](https://mlsanity.com/paper/2007.11142/) (2020)
- [Neural Scene Flow Prior](https://mlsanity.com/paper/2111.01253/) (2021)
- [AFDetV2: Rethinking the Necessity of the Second Stage for Object Detection from Point Clouds](https://mlsanity.com/paper/2112.09205/) (2021)
- [HDNET: Exploiting HD Maps for 3D Object Detection](https://mlsanity.com/paper/2012.11704/) (2020)
- [3D-MAN: 3D Multi-frame Attention Network for Object Detection](https://mlsanity.com/paper/2103.16054/) (2021)
- [MPPNet: Multi-Frame Feature Intertwining with Proxy Points for 3D Temporal Object Detection](https://mlsanity.com/paper/2205.05979/) (2022)
- [AB3DMOT: A Baseline for 3D Multi-Object Tracking and New Evaluation Metrics](https://mlsanity.com/paper/2008.08063/) (2020)
- [Weakly Supervised Learning of Rigid 3D Scene Flow](https://mlsanity.com/paper/2102.08945/) (2021)
- [Bridging the Domain Gap for Multi-Agent Perception](https://mlsanity.com/paper/2210.08451/) (2022)
- [LiDAR Distillation: Bridging the Beam-Induced Domain Gap for 3D Object Detection](https://mlsanity.com/paper/2203.14956/) (2022)
- [Dynamic 3D Scene Analysis by Point Cloud Accumulation](https://mlsanity.com/paper/2207.12394/) (2022)
- [MultiXNet: Multiclass Multistage Multimodal Motion Prediction](https://mlsanity.com/paper/2006.02000/) (2020)
- [Waymo Public Road Safety Performance Data](https://mlsanity.com/paper/2011.00038/) (2020)

## Built on by (most cited first, 6 of 6)

- [Vehicle-to-Everything Cooperative Perception for Autonomous Driving](https://mlsanity.com/paper/2310.03525/) (2023)
- [TraF-Align: Trajectory-aware Feature Alignment for Asynchronous Multi-agent Perception](https://mlsanity.com/paper/2503.19391/) (2025)
- [Mixed Signals: A Diverse Point Cloud Dataset for Heterogeneous LiDAR V2X Collaboration](https://mlsanity.com/paper/2502.14156/) (2025)
- [Systematic Literature Review on Vehicular Collaborative Perception -- A Computer Vision Perspective](https://mlsanity.com/paper/2504.04631/) (2025)
- [End-to-End 3-D Spatiotemporal Perception with Multimodal Fusion and V2X Collaboration](https://mlsanity.com/paper/2512.21831/) (2025)
- [Harnessing Implicit Cooperation: A Multi-Agent Reinforcement Learning Approach Towards Decentralized Local Energy Markets](https://mlsanity.com/paper/2602.16062/) (2026)

## Abstract (arXiv)

Occlusion is a major challenge for LiDAR-based object detection methods. This challenge becomes safety-critical in urban traffic where the ego vehicle must have reliable object detection to avoid collision while its field of view is severely reduced due to the obstruction posed by a large number of road users. Collaborative perception via Vehicle-to-Everything (V2X) communication, which leverages the diverse perspective thanks to the presence at multiple locations of connected agents to form a complete scene representation, is an appealing solution. State-of-the-art V2X methods resolve the performance-bandwidth tradeoff using a mid-collaboration approach where the Bird-Eye View images of point clouds are exchanged so that the bandwidth consumption is lower than communicating point clouds as in early collaboration, and the detection performance is higher than late collaboration, which fuses agents' output, thanks to a deeper interaction among connected agents. While achieving strong performance, the real-world deployment of most mid-collaboration approaches is hindered by their overly complicated architectures, involving learnable collaboration graphs and autoencoder-based compressor/ decompressor, and unrealistic assumptions about inter-agent synchronization. In this work, we devise a simple yet effective collaboration method that achieves a better bandwidth-performance tradeoff than prior state-of-the-art methods while minimizing changes made to the single-vehicle detection models and relaxing unrealistic assumptions on inter-agent synchronization. Experiments on the V2X-Sim dataset show that our collaboration method achieves 98% of the performance of an early-collaboration method, while only consuming the equivalent bandwidth of a late-collaboration method.