mlsanityData to 7 Oct 2026

CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformers

arXiv 2207.02202 ↗First posted 5 Jul 2022cs.CVcited by 227

Not on the map. Why?

Cited by 21 in End-to-end autonomous driving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Bird's eye view (BEV) semantic segmentation plays a crucial role in spatial sensing for autonomous driving. Although recent literature has made significant progress on BEV map understanding, they are all based on single-agent camera-based systems. These solutions sometimes have difficulty handling occlusions or detecting distant objects in complex traffic scenes. Vehicle-to-Vehicle (V2V) communication technologies have enabled autonomous vehicles to share sensing information, dramatically improving the perception performance and range compared to single-agent systems. In this paper, we propose CoBEVT, the first generic multi-agent multi-camera perception framework that can cooperatively generate BEV map predictions. To efficiently fuse camera features from multi-view and multi-agent data in an underlying Transformer architecture, we design a fused axial attention module (FAX), which captures sparsely local and global spatial interactions across views and agents. The extensive experiments on the V2V perception dataset, OPV2V, demonstrate that CoBEVT achieves state-of-the-art performance for cooperative BEV semantic segmentation. Moreover, CoBEVT is shown to be generalizable to other tasks, including 1) BEV segmentation with single-agent multi-camera and 2) 3D object detection with multi-agent LiDAR systems, achieving state-of-the-art performance with real-time inference speed. The code is available at https://github.com/DerrickXuNu/CoBEVT.

Key facts. Cited by 227 papers in the map (data through 7 Oct 2026). Builds on 54 of its 68 references that are in the map. First posted on arXiv 5 Jul 2022.

Builds on

54 of 68 references.

References · 54

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  6. 2016Layer Normalizationcited by 5,579
  7. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  8. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  9. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  10. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  11. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  12. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
38 more
  1. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  2. 2021ViViT: A Video Vision Transformercited by 1,610
  3. 2015Spatial Transformer Networkscited by 1,423
  4. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  5. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,127
  6. 2021Video Swin Transformercited by 1,090
  7. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  8. 2021Multiscale Vision Transformerscited by 896
  9. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 849
  10. 2021CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windowscited by 526
  11. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 516
  12. 2022MaxViT: Multi-Axis Vision Transformercited by 476
  13. 2021Pay Attention to MLPscited by 367
  14. 2021FNet: Mixing Tokens with Fourier Transformscited by 343
  15. 2021XCiT: Cross-Covariance Image Transformerscited by 335
  16. 2020Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentationcited by 326
  17. 2022V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformercited by 320
  18. 2021OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communicationcited by 304
  19. 2022MAXIM: Multi-Axis MLP for Image Processingcited by 301
  20. 2020V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Predictioncited by 292
  21. 2021Focal Self-attention for Local-Global Interactions in Vision Transformerscited by 286
  22. 2022Cross-view Transformers for real-time Map-view Semantic Segmentationcited by 264
  23. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  24. 2021ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Biascited by 216
  25. 2018Orthographic Feature Transform for Monocular 3D Object Detectioncited by 201
  26. 2021Learning Distilled Collaboration Graph for Multi-Agent Perceptioncited by 194
  27. 2019Cross-view Semantic Segmentation for Sensing Surroundingscited by 173
  28. 2019F-Cooper: Feature based Cooperative Perception for Autonomous Vehicle Edge Computing System Using 3D Point Cloudscited by 171
  29. 2019Cooper: Cooperative Perception for Connected Autonomous Vehicles based on 3D Point Cloudscited by 165
  30. 2021Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformercited by 115
  31. 2021Translating Images into Mapscited by 115
  32. 2022Global Context Vision Transformerscited by 98
  33. 2020A Sim2Real Deep Learning Approach for the Transformation of Images from Multiple Vehicle-Mounted Cameras to a Semantically Segmented Image in Bird's Eye Viewcited by 71
  34. 2022Model-Agnostic Multi-Agent Perception Frameworkcited by 51
  35. 2020MonoLayout: Amodal scene layout from a single imagecited by 49
  36. 2021Keypoints-Based Deep Feature Fusion for Cooperative Vehicle Detection of Autonomous Drivingcited by 45
  37. 2020BEV-Seg: Bird's Eye View Semantic Segmentation Using Geometry and Semantic Point Cloudcited by 38
  38. 2019A Geometric Approach to Obtain a Bird's Eye View from an Imagecited by 34

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 14; 2023: 41; 2024: 52; 2025: 63; 2026: 57.

227 papers.

Sort:

By area:

  1. 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 306
  2. 2022Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Mapscited by 255
  3. 2023V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perceptioncited by 155
  4. 2022Robust Collaborative 3D Object Detection in Presence of Pose Errorscited by 129
  5. 2022Vision-Centric BEV Perception: A Surveycited by 126
  6. 2023Street-View Image Generation from a Bird's-Eye View Layoutcited by 102
  7. 2023Collaborative Perception in Autonomous Driving: Methods, Datasets and Challengescited by 87
  8. 2024An Extensible Framework for Open Heterogeneous Collaborative Perceptioncited by 87
  9. 2023Collaboration Helps Camera Overtake LiDAR in 3D Detectioncited by 81
  10. 2023DeepAccident: A Motion and Accident Prediction Benchmark for V2X Autonomous Drivingcited by 73
  11. 2023SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Drivingcited by 72
  12. 2023HM-ViT: Hetero-modal Vehicle-to-Vehicle Cooperative perception with vision transformercited by 70
38 more
  1. 2024Communication-Efficient Collaborative Perception via Information Filling with Codebookcited by 67
  2. 2022Bridging the Domain Gap for Multi-Agent Perceptioncited by 60
  3. 2023Spatio-Temporal Domain Awareness for Multi-Agent Collaborative Perceptioncited by 52
  4. 2022Learning for Vehicle-to-Vehicle Cooperative Perception under Lossy Communicationcited by 47
  5. 2024V2X-VLM: End-to-End V2X Cooperative Autonomous Driving Through Large Vision-Language Modelscited by 47
  6. 2025STAMP: Scalable Task And Model-agnostic Collaborative Perceptioncited by 44
  7. 2024End-to-End Autonomous Driving through V2X Cooperationcited by 42
  8. 2023CORE: Cooperative Reconstruction for Multi-Agent Perceptioncited by 41
  9. 2022X-Align: Cross-Modal Cross-View Alignment for Bird's-Eye-View Segmentationcited by 38
  10. 2023Towards Vehicle-to-everything Autonomous Driving: A Survey on Collaborative Perceptioncited by 38
  11. 2023Among Us: Adversarially Robust Collaborative Perception by Consensuscited by 37
  12. 2024RCooper: A Real-world Large-scale Dataset for Roadside Cooperative Perceptioncited by 37
  13. 2024Collaborative Semantic Occupancy Prediction with Hybrid Feature Fusion in Connected Automated Vehiclescited by 33
  14. 2024Collaborative Perception for Connected and Autonomous Driving: Challenges, Possible Solutions and Opportunitiescited by 32
  15. 2022Adaptive Feature Fusion for Cooperative Perception using LiDAR Point Cloudscited by 29
  16. 2023UMC: A Unified Bandwidth-efficient and Multi-resolution based Collaborative Perception Frameworkcited by 29
  17. 2022V2XP-ASG: Generating Adversarial Scenes for Vehicle-to-Everything Perceptioncited by 28
  18. 2025V2V-LLM: Vehicle-to-Vehicle Cooperative Autonomous Driving with Multimodal Large Language Modelscited by 27
  19. 2025CoSDH: Communication-Efficient Collaborative Perception via Supply-Demand Awareness and Intermediate-Late Hybridizationcited by 27
  20. 2025UniOcc: A Unified Benchmark for Occupancy Forecasting and Prediction in Autonomous Drivingcited by 27
  21. 2024Mask2Map: Vectorized HD Map Construction Using Bird's Eye View Segmentation Maskscited by 26
  22. 2023Grid-Centric Traffic Scenario Perception for Autonomous Driving: A Comprehensive Reviewcited by 25
  23. 2024CMP: Cooperative Motion Prediction with Multi-Agent Communicationcited by 25
  24. 2025LangCoop: Collaborative Driving with Languagecited by 25
  25. 2022A Survey and Framework of Cooperative Perception: From Heterogeneous Singleton to Hierarchical Cooperationcited by 21
  26. 2023Learning Cooperative Trajectory Representations for Motion Forecastingcited by 21
  27. 2024V2X-Radar: A Multi-modal Dataset with 4D Radar for Cooperative Perceptioncited by 21
  28. 2022BEV-SAN: Accurate BEV 3D Object Detection via Slice Attention Networkscited by 20
  29. 2023BM2CP: Efficient Collaborative Perception with LiDAR-Camera Modalitiescited by 20
  30. 2024Towards Collaborative Autonomous Driving: Simulation Platform and End-to-End Systemcited by 20
  31. 2023Adaptive Communications in Collaborative Perception with Domain Alignment for Autonomous Drivingcited by 19
  32. 2023S2R-ViT for Multi-Agent Cooperative Perception: Bridging the Gap from Simulation to Realitycited by 18
  33. 2024V2X-R: Cooperative LiDAR-4D Radar Fusion with Denoising Diffusion for 3D Object Detectioncited by 18
  34. 2023DUSA: Decoupled Unsupervised Sim2Real Adaptation for Vehicle-to-Everything Collaborative Perceptioncited by 17
  35. 2023AccidentGPT: Accident Analysis and Prevention from V2X Environmental Perception with Multi-modal Large Modelcited by 17
  36. 2023ScalableMap: Scalable Map Learning for Online Long-Range Vectorized HD Map Constructioncited by 16
  37. 2023Mitigating Transformer Overconfidence via Lipschitz Regularizationcited by 15
  38. 2023Probabilistic 3D Multi-Object Cooperative Tracking for Autonomous Driving via Differentiable Multi-Sensor Kalman Filtercited by 15

50 most cited of 227 All 227 (JSON)