# Spatio-Temporal Domain Awareness for Multi-Agent Collaborative Perception

arXiv: https://arxiv.org/abs/2307.13929 · HTML: https://mlsanity.com/paper/2307.13929/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 49 references that are in the map.
- First posted on arXiv 26 Jul 2023.

## Builds on (most cited first, 30 of 35)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Learning to Communicate with Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1605.06676/) (2016)
- [Learning Multiagent Communication with Backpropagation](https://mlsanity.com/paper/1605.07736/) (2016)
- [V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer](https://mlsanity.com/paper/2203.10638/) (2022)
- [OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communication](https://mlsanity.com/paper/2109.07644/) (2021)
- [V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction](https://mlsanity.com/paper/2008.07519/) (2020)
- [DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detection](https://mlsanity.com/paper/2204.05575/) (2022)
- [Deep Continuous Fusion for Multi-Sensor 3D Object Detection](https://mlsanity.com/paper/2012.10992/) (2020)
- [Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Maps](https://mlsanity.com/paper/2209.12836/) (2022)
- [CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformers](https://mlsanity.com/paper/2207.02202/) (2022)
- [TarMAC: Targeted Multi-Agent Communication](https://mlsanity.com/paper/1810.11187/) (2018)
- [Learning Distilled Collaboration Graph for Multi-Agent Perception](https://mlsanity.com/paper/2111.00643/) (2021)
- [Learning Attentional Communication for Multi-Agent Cooperation](https://mlsanity.com/paper/1805.07733/) (2018)
- [F-Cooper: Feature based Cooperative Perception for Autonomous Vehicle Edge Computing System Using 3D Point Clouds](https://mlsanity.com/paper/1909.06459/) (2019)
- [Cooper: Cooperative Perception for Connected Autonomous Vehicles based on 3D Point Clouds](https://mlsanity.com/paper/1905.05265/) (2019)
- [When2com: Multi-Agent Perception via Communication Graph Grouping](https://mlsanity.com/paper/2006.00176/) (2020)
- [V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perception](https://mlsanity.com/paper/2303.07601/) (2023)
- [Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks](https://mlsanity.com/paper/1812.09755/) (2018)
- [CenterFormer: Center-based Transformer for 3D Object Detection](https://mlsanity.com/paper/2209.05588/) (2022)
- [Latency-Aware Collaborative Perception](https://mlsanity.com/paper/2207.08560/) (2022)
- [Collaboration Helps Camera Overtake LiDAR in 3D Detection](https://mlsanity.com/paper/2303.13560/) (2023)
- [Bridging the Domain Gap for Multi-Agent Perception](https://mlsanity.com/paper/2210.08451/) (2022)
- [Temporal-Channel Transformer for 3D Lidar-Based Video Object Detection in Autonomous Driving](https://mlsanity.com/paper/2011.13628/) (2020)
- [VAIN: Attentional Multi-agent Predictive Modeling](https://mlsanity.com/paper/1706.06122/) (2017)
- [Model-Agnostic Multi-Agent Perception Framework](https://mlsanity.com/paper/2203.13168/) (2022)
- [Learning for Vehicle-to-Vehicle Cooperative Perception under Lossy Communication](https://mlsanity.com/paper/2212.08273/) (2022)

## Built on by (most cited first, 30 of 52)

- [Communication-Efficient Collaborative Perception via Information Filling with Codebook](https://mlsanity.com/paper/2405.04966/) (2024)
- [Asynchrony-Robust Collaborative Perception via Bird's Eye View Flow](https://mlsanity.com/paper/2309.16940/) (2023)
- [End-to-End Autonomous Driving through V2X Cooperation](https://mlsanity.com/paper/2404.00717/) (2024)
- [Towards Vehicle-to-everything Autonomous Driving: A Survey on Collaborative Perception](https://mlsanity.com/paper/2308.16714/) (2023)
- [Collaborative Semantic Occupancy Prediction with Hybrid Feature Fusion in Connected Automated Vehicles](https://mlsanity.com/paper/2402.07635/) (2024)
- [Correlation-Decoupled Knowledge Distillation for Multimodal Sentiment Analysis with Incomplete Modalities](https://mlsanity.com/paper/2404.16456/) (2024)
- [V2XPnP: Vehicle-to-Everything Spatio-Temporal Fusion for Multi-Agent Perception and Prediction](https://mlsanity.com/paper/2412.01812/) (2024)
- [Large Model Based Agents: State-of-the-Art, Cooperation Paradigms, Security and Privacy, and Future Trends](https://mlsanity.com/paper/2409.14457/) (2024)
- [Robust Emotion Recognition in Context Debiasing](https://mlsanity.com/paper/2403.05963/) (2024)
- [Networking Systems for Video Anomaly Detection: A Tutorial and Survey](https://mlsanity.com/paper/2405.10347/) (2024)
- [V2X-R: Cooperative LiDAR-4D Radar Fusion with Denoising Diffusion for 3D Object Detection](https://mlsanity.com/paper/2411.08402/) (2024)
- [Vehicle-to-Everything Cooperative Perception for Autonomous Driving](https://mlsanity.com/paper/2310.03525/) (2023)
- [One is Plenty: A Polymorphic Feature Interpreter for Immutable Heterogeneous Collaborative Perception](https://mlsanity.com/paper/2411.16799/) (2024)
- [TraF-Align: Trajectory-aware Feature Alignment for Asynchronous Multi-agent Perception](https://mlsanity.com/paper/2503.19391/) (2025)
- [UrbanIng-V2X: A Large-Scale Multi-Vehicle, Multi-Infrastructure Dataset Across Multiple Intersections for Cooperative Perception](https://mlsanity.com/paper/2510.23478/) (2025)
- [Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directions](https://mlsanity.com/paper/2503.03262/) (2025)
- [A Survey on Intermediate Fusion Methods for Collaborative Perception Categorized by Real World Challenges](https://mlsanity.com/paper/2404.16139/) (2024)
- [DSRC: Learning Density-insensitive and Semantic-aware Collaborative Representation against Corruptions](https://mlsanity.com/paper/2412.10739/) (2024)
- [Generative Multi-Agent Collaboration in Embodied AI: A Systematic Review](https://mlsanity.com/paper/2502.11518/) (2025)
- [V2X-DGW: Domain Generalization for Multi-agent Perception under Adverse Weather Conditions](https://mlsanity.com/paper/2403.11371/) (2024)
- [Drones Help Drones: A Collaborative Framework for Multi-Drone Object Trajectory Prediction and Beyond](https://mlsanity.com/paper/2405.14674/) (2024)
- [Towards Context-Aware Emotion Recognition Debiasing from a Causal Demystification Perspective via De-confounded Training](https://mlsanity.com/paper/2407.04963/) (2024)
- [Pragmatic Communication in Multi-Agent Collaborative Perception](https://mlsanity.com/paper/2401.12694/) (2024)
- [Debiased Multimodal Understanding for Human Language Sequences](https://mlsanity.com/paper/2403.05025/) (2024)
- [V2X-PC: Vehicle-to-everything Collaborative Perception via Point Cluster](https://mlsanity.com/paper/2403.16635/) (2024)
- [CoMamba: Real-time Cooperative Perception Unlocked with State Space Models](https://mlsanity.com/paper/2409.10699/) (2024)
- [mmCooper: A Multi-agent Multi-stage Communication-efficient and Collaboration-robust Cooperative Perception Framework](https://mlsanity.com/paper/2501.12263/) (2025)
- [TurboTrain: Towards Efficient and Balanced Multi-Task Learning for Multi-Agent Perception and Prediction](https://mlsanity.com/paper/2508.04682/) (2025)
- [SiCP: Simultaneous Individual and Cooperative Perception for 3D Object Detection in Connected and Automated Vehicles](https://mlsanity.com/paper/2312.04822/) (2023)
- [CoDynTrust: Robust Asynchronous Collaborative Perception via Dynamic Feature Trust Modulus](https://mlsanity.com/paper/2502.08169/) (2025)

## Abstract (arXiv)

Multi-agent collaborative perception as a potential application for vehicle-to-everything communication could significantly improve the perception performance of autonomous vehicles over single-agent perception. However, several challenges remain in achieving pragmatic information sharing in this emerging research. In this paper, we propose SCOPE, a novel collaborative perception framework that aggregates the spatio-temporal awareness characteristics across on-road agents in an end-to-end manner. Specifically, SCOPE has three distinct strengths: i) it considers effective semantic cues of the temporal context to enhance current representations of the target agent; ii) it aggregates perceptually critical spatial information from heterogeneous agents and overcomes localization errors via multi-scale feature interactions; iii) it integrates multi-source representations of the target agent based on their complementary contributions by an adaptive fusion paradigm. To thoroughly evaluate SCOPE, we consider both real-world and simulated scenarios of collaborative 3D object detection tasks on three datasets. Extensive experiments demonstrate the superiority of our approach and the necessity of the proposed components.