mlsanityData to 14 Oct 2026

Bridging Perspectives: Foundation Model Guided BEV Maps for 3D Object Detection and Tracking

arXiv 2510.10287 ↗First posted 11 Oct 2025cs.CV cs.ROcited by 9

Not on the map. Why?

Abstract (arXiv)

Camera-based 3D object detection and tracking are essential for perception in autonomous driving. Current state-of-the-art approaches often rely exclusively on either perspective-view (PV) or bird's-eye-view (BEV) features, limiting their ability to leverage both fine-grained object details and spatially structured scene representations. In this work, we propose DualViewDistill, a hybrid detection and tracking framework that incorporates both PV and BEV camera image features to leverage their complementary strengths. Our approach introduces BEV maps guided by foundation models, leveraging descriptive DINOv2 features that are distilled into BEV representations through a novel distillation process. By integrating PV features with BEV maps enriched with semantic and geometric features from DINOv2, our model leverages this hybrid representation via deformable aggregation to enhance 3D object detection and tracking. Extensive experiments on the nuScenes and Argoverse 2 benchmarks demonstrate that DualViewDistill achieves state-of-the-art performance. The results showcase the potential of foundation model BEV maps to enable more reliable perception for autonomous driving. We make the code and pre-trained models available at https://dualviewdistill.cs.uni-freiburg.de .

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 63 of its 71 references that are in the map. First posted on arXiv 11 Oct 2025.

Builds on

63 of 71 references.

References · 63

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  6. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  7. 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
  8. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  9. 2020Center-based 3D Object Detection and Trackingcited by 1,144
  10. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,044
  11. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 850
  12. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 713
38 more
  1. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 707
  2. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 572
  3. 2022Vision Transformer Adapter for Dense Predictionscited by 565
  4. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 552
  5. 2021TrackFormer: Multi-Object Tracking with Transformerscited by 518
  6. 2021MOTR: End-to-End Multiple-Object Tracking with Transformercited by 381
  7. 2023EVA-02: A Visual Representation for Neon Genesiscited by 381
  8. 2019Class-balanced Grouping and Sampling for Point Cloud 3D Object Detectioncited by 350
  9. 2022BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detectioncited by 341
  10. 2023Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detectioncited by 315
  11. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
  12. 2025MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Detailscited by 261
  13. 2021Is Pseudo-Lidar needed for Monocular 3D Object detection?cited by 249
  14. 2024SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representationcited by 241
  15. 2022Time Will Tell: New Outlooks and A Baseline for Temporal Multi-View 3D Object Detectioncited by 178
  16. 2019An Energy and GPU-Computation Efficient Backbone Network for Real-Time Object Detectioncited by 163
  17. 2022Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusioncited by 162
  18. 2023SparseBEV: High-Performance Sparse 3D Object Detection from Multi-Camera Videoscited by 150
  19. 2023Sparse4D v2: Recurrent Temporal Fusion with Sparse Modelcited by 93
  20. 2022MUTR3D: A Multi-camera Tracking Framework via 3D-to-2D Queriescited by 91
  21. 2022BEVPoolv2: A Cutting-edge Implementation of BEVDet Toward Deploymentcited by 79
  22. 2020HDNET: Exploiting HD Maps for 3D Object Detectioncited by 78
  23. 2023Far3D: Expanding the Horizon for Surround-view 3D Object Detectioncited by 70
  24. 2023Sparse4D v3: Advancing End-to-End 3D Detection and Trackingcited by 70
  25. 2023Neural Map Prior for Autonomous Drivingcited by 68
  26. 2023Exploring Recurrent Long-term Temporal Fusion for Multi-view 3D Perceptioncited by 65
  27. 2023Standing Between Past and Future: Spatio-Temporal Modeling for Multi-Camera 3D Multi-Object Trackingcited by 54
  28. 2024Lexicon3D: Probing Visual Foundation Models for Complex 3D Scene Understandingcited by 49
  29. 2024SparseAD: Sparse Query-Centric Paradigm for Efficient End-to-End Autonomous Drivingcited by 47
  30. 2023MOTRv3: Release-Fetch Supervision for End-to-End Multi-Object Trackingcited by 42
  31. 2024Ray Denoising: Depth-aware Hard Negative Sampling for Multi-view 3D Object Detectioncited by 36
  32. 2023BEVNeXt: Reviving Dense BEV Frameworks for 3D Object Detectioncited by 35
  33. 2024ADA-Track++: End-to-End Multi-Camera 3D Multi-Object Tracking with Alternating Detection and Associationcited by 33
  34. 2024DistillNeRF: Perceiving 3D Scenes from Single-Glance Images by Distilling Neural Fields and Foundation Model Featurescited by 33
  35. 20223D Multi-Object Tracking Using Graph Neural Networks with Cross-Edge Modality Attentioncited by 27
  36. 2024BEVCar: Camera-Radar Fusion for BEV Map and Object Segmentationcited by 27
  37. 2024MV2DFusion: Leveraging Modality-Specific Object Semantics for Multi-Modal 3D Detectioncited by 23
  38. 2024PreSight: Enhancing Autonomous Vehicle Perception with City-Scale NeRF Priorscited by 21

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 9.

9 papers.

  1. 2026Latent Gaussian Splatting for 4D Panoptic Occupancy Trackingcited by 3
  2. 2026FishRoPE: Projective Rotary Position Embeddings for Omnidirectional Visual Perceptioncited by 3
  3. 2026Sparse3DTrack: Monocular 3D Object Tracking Using Sparse Supervisioncited by 2
  4. 2026Leveraging Previous-Traversal Point Cloud Map Priors for Camera-Based 3D Object Detection and Trackingcited by 2
  5. 2026Fusion-Poly: A Polyhedral Framework Based on Spatial-Temporal Fusion for 3D Multi-Object Trackingcited by 1
  6. 2026GhostPoint: Self-Supervised Representation Learning by Hallucinating Occluded LiDAR Structurecited by 1
  7. 2026CAM3DNet: Comprehensively mining the multi-scale features for 3D Object Detection with Multi-View Camerascited by 0
  8. 2026SimPB++: Simultaneously Detecting 2D and 3D Objects from Multiple Camerascited by 0
  9. 2026Leveraging Vision-Based Point Cloud Map Priors for Camera-Based 3D Object Detection and Online Vectorized HD Mappingcited by 0