mlsanityData to 7 Oct 2026

CLIP-BEVFormer: Enhancing Multi-View Image-Based BEV Detector with Ground Truth Flow

arXiv 2403.08919 ↗First posted 13 Mar 2024cs.CVcited by 14

Not on the map. Why?

Abstract (arXiv)

Autonomous driving stands as a pivotal domain in computer vision, shaping the future of transportation. Within this paradigm, the backbone of the system plays a crucial role in interpreting the complex environment. However, a notable challenge has been the loss of clear supervision when it comes to Bird's Eye View elements. To address this limitation, we introduce CLIP-BEVFormer, a novel approach that leverages the power of contrastive learning techniques to enhance the multi-view image-derived BEV backbones with ground truth information flow. We conduct extensive experiments on the challenging nuScenes dataset and showcase significant and consistent improvements over the SOTA. Specifically, CLIP-BEVFormer achieves an impressive 8.5% and 9.2% enhancement in terms of NDS and mAP, respectively, over the previous best BEV model on the 3D object detection task.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 40 of its 49 references that are in the map. First posted on arXiv 13 Mar 2024.

Builds on

40 of 49 references.

References · 40

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2020End-to-End Object Detection with Transformerscited by 7,384
  6. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  7. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  8. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  9. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  10. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  11. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,427
  12. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
28 more
  1. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  2. 2022Planning-oriented Autonomous Drivingcited by 1,068
  3. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  4. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,021
  5. 2021Florence: A New Foundation Model for Computer Visioncited by 802
  6. 2021Robust fine-tuning of zero-shot modelscited by 785
  7. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 721
  8. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 655
  9. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 646
  10. 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 636
  11. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 613
  12. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  13. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
  14. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 444
  15. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modelingcited by 429
  16. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 365
  17. 2021ActionCLIP: A New Paradigm for Video Action Recognitioncited by 317
  18. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
  19. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  20. 2022BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Drivingcited by 191
  21. 2021Probabilistic and Geometric Depth: Detecting Objects in Perspectivecited by 190
  22. 2019Cross-view Semantic Segmentation for Sensing Surroundingscited by 173
  23. 2022Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusioncited by 161
  24. 2022PersFormer: 3D Lane Detection via Perspective Transformer and the OpenLane Benchmarkcited by 140
  25. 2024VLP: Vision Language Planning for Autonomous Drivingcited by 125
  26. 2023Sparse4D v2: Recurrent Temporal Fusion with Sparse Modelcited by 92
  27. 2020SSN: Shape Signature Networks for Multi-class Object Detection from Point Cloudscited by 74
  28. 2023Sparse4D v3: Advancing End-to-End 3D Detection and Trackingcited by 69

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 8; 2026: 1.

14 papers.

Sort:

By area:

  1. 2025WOD-E2E: Waymo Open Dataset for End-to-End Driving in Challenging Long-tail Scenarioscited by 60
  2. 2024MaskBEV: Towards A Unified Framework for BEV Detection and Map Segmentationcited by 10
  3. 2024Feature Map Convergence Evaluation for Functional Modulecited by 6
  4. 2024CLIP meets DINO for Tuning Zero-Shot Classifier using Unlabeled Image Collectionscited by 5
  5. 2025Divide and Merge: Motion and Semantic Learning in End-to-End Autonomous Drivingcited by 5
  6. 2025Foundation Models for Trajectory Planning in Autonomous Driving: A Review of Progress and Open Challengescited by 3
  7. 2025DGFusion: Dual-guided Fusion for Robust Multi-Modal 3D Object Detectioncited by 2
  8. 2024MonoDETRNext: Next-Generation Accurate and Efficient Monocular 3D Object Detectorcited by 1
  9. 2024Unveiling the Black Box: Independent Functional Module Evaluation for Bird's-Eye-View Perception Modelcited by 1
  10. 2025Hierarchical Feature-level Reverse Propagation for Post-Training Neural Networkscited by 0
  11. 2025Decoupled Functional Evaluation of Autonomous Driving Models via Feature Map Quality Scoringcited by 0
  12. 2025Refine-and-Contrast: Adaptive Instance-Aware BEV Representations for Multi-UAV Collaborative Object Detectioncited by 0
2 more
  1. 2025StereoMV2D: A Sparse Temporal Stereo-Enhanced Framework for Robust Multi-View 3D Object Detectioncited by 0
  2. 2026Toward Unified Multimodal Representation Learning for Autonomous Drivingcited by 0