# RoCo:Robust Collaborative Perception By Iterative Object Matching and Pose Adjustment

arXiv: https://arxiv.org/abs/2408.00257 · HTML: https://mlsanity.com/paper/2408.00257/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 49 references that are in the map.
- First posted on arXiv 1 Aug 2024.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer](https://mlsanity.com/paper/2203.10638/) (2022)
- [V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction](https://mlsanity.com/paper/2008.07519/) (2020)
- [Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Maps](https://mlsanity.com/paper/2209.12836/) (2022)
- [CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformers](https://mlsanity.com/paper/2207.02202/) (2022)
- [F-Cooper: Feature based Cooperative Perception for Autonomous Vehicle Edge Computing System Using 3D Point Clouds](https://mlsanity.com/paper/1909.06459/) (2019)
- [V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Driving](https://mlsanity.com/paper/2202.08449/) (2022)
- [Robust Collaborative 3D Object Detection in Presence of Pose Errors](https://mlsanity.com/paper/2211.07214/) (2022)
- [An Extensible Framework for Open Heterogeneous Collaborative Perception](https://mlsanity.com/paper/2401.13964/) (2024)
- [3D Point Cloud Processing and Learning for Autonomous Driving](https://mlsanity.com/paper/2003.00601/) (2020)
- [Asynchrony-Robust Collaborative Perception via Bird's Eye View Flow](https://mlsanity.com/paper/2309.16940/) (2023)
- [Learning to Communicate and Correct Pose Errors](https://mlsanity.com/paper/2011.05289/) (2020)
- [DSP-SLAM: Object Oriented SLAM with Deep Shape Priors](https://mlsanity.com/paper/2108.09481/) (2021)
- [V2XP-ASG: Generating Adversarial Scenes for Vehicle-to-Everything Perception](https://mlsanity.com/paper/2209.13679/) (2022)
- [DUSA: Decoupled Unsupervised Sim2Real Adaptation for Vehicle-to-Everything Collaborative Perception](https://mlsanity.com/paper/2310.08117/) (2023)
- [Overcoming Obstructions via Bandwidth-Limited Multi-Agent Spatial Handshaking](https://mlsanity.com/paper/2107.00771/) (2021)
- [SyncNet: correlating objective for time delay estimation in audio signals](https://mlsanity.com/paper/2203.14639/) (2022)
- [Multiple Hypothesis Semantic Mapping for Robust Data Association](https://mlsanity.com/paper/2012.04423/) (2020)
- [MOTSLAM: MOT-assisted monocular dynamic SLAM using single-view depth estimation](https://mlsanity.com/paper/2210.02038/) (2022)
- [VOLoc: Visual Place Recognition by Querying Compressed Lidar Map](https://mlsanity.com/paper/2402.15961/) (2024)
- [DL-SLOT: Dynamic LiDAR SLAM and object tracking based on collaborative graph optimization](https://mlsanity.com/paper/2212.02077/) (2022)

## Built on by (most cited first, 12 of 12)

- [MambaVO: Deep Visual Odometry Based on Sequential Matching Refinement and Training Smoothing](https://mlsanity.com/paper/2412.20082/) (2024)
- [CoDiff: Conditional Diffusion Model for Collaborative 3D Object Detection](https://mlsanity.com/paper/2502.14891/) (2025)
- [Point-Cache: Test-time Dynamic and Hierarchical Cache for Robust and Generalizable Point Cloud Analysis](https://mlsanity.com/paper/2503.12150/) (2025)
- [EIMC: Efficient Instance-aware Multi-modal Collaborative Perception](https://mlsanity.com/paper/2603.02532/) (2026)
- [CoDS: Enhancing Collaborative Perception in Heterogeneous Scenarios via Domain Separation](https://mlsanity.com/paper/2510.13432/) (2025)
- [BOLT: Online Lightweight Adaptation for Preparation-Free Heterogeneous Cooperative Perception](https://mlsanity.com/paper/2605.00405/) (2026)
- [CATNet: Collaborative Alignment and Transformation Network for Cooperative Perception](https://mlsanity.com/paper/2603.05255/) (2026)
- [Adaptation-Free Heterogeneous Collaborative Perception with Unseen Agent Configurations](https://mlsanity.com/paper/2605.26642/) (2026)
- [UECP: Uncertainty-Enhanced Collaborative Perception](https://mlsanity.com/paper/2606.23046/) (2026)
- [HeteroPROMPT: A Real-time and Privacy-Preserving Heterogeneous Collaborative Perception Framework](https://mlsanity.com/paper/2607.26283/) (2026)
- [CoAnchor: Robust Collaborative Perception under Spatio-Temporal Misalignment via Object-Level Anchors](https://mlsanity.com/paper/2608.21055/) (2026)
- [CoLMIN: LLM-based Multi-Decision Path Negotiation for Cooperative Autonomous Driving](https://mlsanity.com/paper/2609.04807/) (2026)

## Abstract (arXiv)

Collaborative autonomous driving with multiple vehicles usually requires the data fusion from multiple modalities. To ensure effective fusion, the data from each individual modality shall maintain a reasonably high quality. However, in collaborative perception, the quality of object detection based on a modality is highly sensitive to the relative pose errors among the agents. It leads to feature misalignment and significantly reduces collaborative performance. To address this issue, we propose RoCo, a novel unsupervised framework to conduct iterative object matching and agent pose adjustment. To the best of our knowledge, our work is the first to model the pose correction problem in collaborative perception as an object matching task, which reliably associates common objects detected by different agents. On top of this, we propose a graph optimization process to adjust the agent poses by minimizing the alignment errors of the associated objects, and the object matching is re-done based on the adjusted agent poses. This process is carried out iteratively until convergence. Experimental study on both simulated and real-world datasets demonstrates that the proposed framework RoCo consistently outperforms existing relevant methods in terms of the collaborative object detection performance, and exhibits highly desired robustness when the pose information of agents is with high-level noise. Ablation studies are also provided to show the impact of its key parameters and components. The code is released at https://github.com/HuangZhe885/RoCo.