mlsanityData to 7 Oct 2026

Collaboration Helps Camera Overtake LiDAR in 3D Detection

arXiv 2303.13560 ↗First posted 23 Mar 2023cs.CVcited by 81

Not on the map. Why?

Abstract (arXiv)

Camera-only 3D detection provides an economical solution with a simple configuration for localizing objects in 3D space compared to LiDAR-based detection systems. However, a major challenge lies in precise depth estimation due to the lack of direct 3D measurements in the input. Many previous methods attempt to improve depth estimation through network designs, e.g., deformable layers and larger receptive fields. This work proposes an orthogonal direction, improving the camera-only 3D detection by introducing multi-agent collaborations. Our proposed collaborative camera-only 3D detection (CoCa3D) enables agents to share complementary information with each other through communication. Meanwhile, we optimize communication efficiency by selecting the most informative cues. The shared messages from multiple viewpoints disambiguate the single-agent estimated depth and complement the occluded and long-range regions in the single-agent view. We evaluate CoCa3D in one real-world dataset and two new simulation datasets. Results show that CoCa3D improves previous SOTA performances by 44.21% on DAIR-V2X, 30.60% on OPV2V+, 12.59% on CoPerception-UAVs+ for AP@70. Our preliminary results show a potential that with sufficient collaboration, the camera might overtake LiDAR in some practical scenarios. We released the dataset and code at https://siheng-chen.github.io/dataset/CoPerception+ and https://github.com/MediaBrain-SJTU/CoCa3D.

Key facts. Cited by 81 papers in the map (data through 7 Oct 2026). Builds on 41 of its 47 references that are in the map. First posted on arXiv 23 Mar 2023.

Builds on

41 of 47 references.

References · 41

  1. 2020End-to-End Object Detection with Transformerscited by 7,384
  2. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  3. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  4. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  5. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  6. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  7. 2019Objects as Pointscited by 1,172
  8. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  9. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  10. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 890
  11. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
  12. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 711
29 more
  1. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 655
  2. 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 517
  3. 20203DSSD: Point-based 3D Single Stage Object Detectorcited by 440
  4. 2020Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detectioncited by 409
  5. 2022BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detectioncited by 340
  6. 2021Categorical Depth Distribution Network for Monocular 3D Object Detectioncited by 326
  7. 2022V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformercited by 320
  8. 2021OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communicationcited by 304
  9. 2020V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Predictioncited by 292
  10. 2022DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detectioncited by 270
  11. 2022Cross-view Transformers for real-time Map-view Semantic Segmentationcited by 264
  12. 2022Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Mapscited by 255
  13. 2021PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detectioncited by 232
  14. 2022CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformerscited by 227
  15. 2018Orthographic Feature Transform for Monocular 3D Object Detectioncited by 201
  16. 2021Learning Distilled Collaboration Graph for Multi-Agent Perceptioncited by 194
  17. 2022BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Drivingcited by 191
  18. 2020When2com: Multi-Agent Perception via Communication Graph Groupingcited by 157
  19. 2022V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Drivingcited by 157
  20. 2022Robust Collaborative 3D Object Detection in Presence of Pose Errorscited by 129
  21. 2020Who2com: Collaborative Perception via Learnable Handshake Communicationcited by 127
  22. 2022BEVSegFormer: Bird's Eye View Semantic Segmentation From Arbitrary Camera Rigscited by 123
  23. 2020CIA-SSD: Confident IoU-Aware Single-Stage Object Detector From Point Cloudcited by 107
  24. 2020MotionNet: Joint Perception and Motion Prediction for Autonomous Driving Based on Bird's Eye View Mapscited by 100
  25. 2022Latency-Aware Collaborative Perceptioncited by 86
  26. 2021Pyramid R-CNN: Towards Better Performance and Adaptability for 3D Object Detectioncited by 79
  27. 20203D Point Cloud Processing and Learning for Autonomous Drivingcited by 68
  28. 2022Monocular 3D Object Detection with Depth from Motioncited by 38
  29. 2022Aerial Monocular 3D Object Detectioncited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 10; 2024: 23; 2025: 32; 2026: 16.

81 papers.

Sort:

By area:

  1. 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Surveycited by 101
  2. 2024An Extensible Framework for Open Heterogeneous Collaborative Perceptioncited by 87
  3. 2023SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Drivingcited by 72
  4. 2024Communication-Efficient Collaborative Perception via Information Filling with Codebookcited by 67
  5. 2024TUMTraf V2X Cooperative Perception Datasetcited by 66
  6. 2023Spatio-Temporal Domain Awareness for Multi-Agent Collaborative Perceptioncited by 52
  7. 2024V2X-VLM: End-to-End V2X Cooperative Autonomous Driving Through Large Vision-Language Modelscited by 47
  8. 2024End-to-End Autonomous Driving through V2X Cooperationcited by 42
  9. 2023Towards Vehicle-to-everything Autonomous Driving: A Survey on Collaborative Perceptioncited by 38
  10. 2024RCooper: A Real-world Large-scale Dataset for Roadside Cooperative Perceptioncited by 37
  11. 2024Collaborative Semantic Occupancy Prediction with Hybrid Feature Fusion in Connected Automated Vehiclescited by 33
  12. 2023Flow-Based Feature Fusion for Vehicle-Infrastructure Cooperative 3D Object Detectioncited by 31
38 more
  1. 2025LangCoop: Collaborative Driving with Languagecited by 25
  2. 2023QUEST: Query Stream for Practical Cooperative Perceptioncited by 23
  3. 2023Learning Cooperative Trajectory Representations for Motion Forecastingcited by 21
  4. 2023BM2CP: Efficient Collaborative Perception with LiDAR-Camera Modalitiescited by 20
  5. 2024Towards Collaborative Autonomous Driving: Simulation Platform and End-to-End Systemcited by 20
  6. 2024Multiagent Multitraversal Multimodal Self-Driving: Open MARS Datasetcited by 18
  7. 2023Vehicle-to-Everything Cooperative Perception for Autonomous Drivingcited by 15
  8. 2025Griffin: Aerial-Ground Cooperative Detection and Tracking Dataset and Benchmarkcited by 15
  9. 2024A Survey on Intermediate Fusion Methods for Collaborative Perception Categorized by Real World Challengescited by 13
  10. 2024Drones Help Drones: A Collaborative Framework for Multi-Drone Object Trajectory Prediction and Beyondcited by 12
  11. 2025CoopTrack: Exploring End-to-End Learning for Efficient Cooperative Sequential Perceptioncited by 12
  12. 2025AirV2X: Unified Air-Ground Vehicle-to-Everything Collaborationcited by 11
  13. 2025CRUISE: Cooperative Reconstruction and Editing in V2X Scenarios using Gaussian Splattingcited by 11
  14. 2024Pragmatic Communication in Multi-Agent Collaborative Perceptioncited by 10
  15. 2024EMIFF: Enhanced Multi-scale Image Feature Fusion for Vehicle-Infrastructure Cooperative 3D Object Detectioncited by 9
  16. 2024Adver-City: Open-Source Multi-Modal Dataset for Collaborative Perception Under Adverse Weather Conditionscited by 9
  17. 2025CoopDETR: A Unified Cooperative Perception Framework for 3D Detection via Object Querycited by 9
  18. 2025Pragmatic Heterogeneous Collaborative Perception via Generative Communication Mechanismcited by 9
  19. 2025Collaborative Perception Datasets for Autonomous Driving: A Reviewcited by 8
  20. 2024ActFormer: Scalable Collaborative Perception via Active Queriescited by 7
  21. 2024UVCPNet: A UAV-Vehicle Collaborative Perception Network for 3D Object Detectioncited by 7
  22. 2025Learning to Detect Objects from Multi-Agent LiDAR Scans without Manual Labelscited by 7
  23. 2024RCDN: Towards Robust Camera-Insensitivity Collaborative Perception via Dynamic Feature-based 3D Neural Modelingcited by 6
  24. 2024DiffCP: Ultra-Low Bit Collaborative Perception via Diffusion Modelcited by 6
  25. 2025Is Discretization Fusion All You Need for Collaborative Perception?cited by 6
  26. 2023V2X-Boosted Federated Learning for Cooperative Intelligent Transportation Systems with Contextual Client Selectioncited by 5
  27. 2025Systematic Literature Review on Vehicular Collaborative Perception -- A Computer Vision Perspectivecited by 5
  28. 2025A Survey of Multi-sensor Fusion Perception for Embodied AI: Background, Methods, Challenges and Prospectscited by 5
  29. 2025NegoCollab: A Common Representation Negotiation Approach for Heterogeneous Collaborative Perceptioncited by 5
  30. 2025LaneDiffusion: Improving Centerline Graph Learning via Prior Injected BEV Feature Generationcited by 5
  31. 2024V2VSSC: A 3D Semantic Scene Completion Benchmark for Perception with Vehicle to Vehicle Communicationcited by 4
  32. 2024RS-DFM: A Remote Sensing Distributed Foundation Model for Diverse Downstream Taskscited by 4
  33. 2025Which2comm: An Efficient Collaborative Perception Framework for 3D Object Detectioncited by 4
  34. 2025SparseCoop: Cooperative Perception with Kinematic-Grounded Queriescited by 4
  35. 2024Unlocking Past Information: Temporal Embeddings in Cooperative Bird's Eye View Predictioncited by 3
  36. 2024WHALES: A Multi-Agent Scheduling Dataset for Enhanced Cooperation in Autonomous Drivingcited by 3
  37. 2025Vision-Only Gaussian Splatting for Collaborative Semantic Occupancy Predictioncited by 3
  38. 2025U2UData+: A Scalable Swarm UAVs Autonomous Flight Dataset for Embodied Long-horizon Taskscited by 3

50 most cited of 81 All 81 (JSON)