mlsanityData to 7 Oct 2026

Time Will Tell: New Outlooks and A Baseline for Temporal Multi-View 3D Object Detection

arXiv 2210.02443 ↗First posted 5 Oct 2022cs.CV cs.AI cs.LGcited by 177

Not on the map. Why?

Abstract (arXiv)

While recent camera-only 3D detection methods leverage multiple timesteps, the limited history they use significantly hampers the extent to which temporal fusion can improve object perception. Observing that existing works' fusion of multi-frame images are instances of temporal stereo matching, we find that performance is hindered by the interplay between 1) the low granularity of matching resolution and 2) the sub-optimal multi-view setup produced by limited history usage. Our theoretical and empirical analysis demonstrates that the optimal temporal difference between views varies significantly for different pixels and depths, making it necessary to fuse many timesteps over long-term history. Building on our investigation, we propose to generate a cost volume from a long history of image observations, compensating for the coarse but efficient matching resolution with a more optimal multi-view matching setup. Further, we augment the per-frame monocular depth predictions used for long-term, coarse matching with short-term, fine-grained matching and find that long and short term temporal fusion are highly complementary. While maintaining high efficiency, our framework sets new state-of-the-art on nuScenes, achieving first place on the test set and outperforming previous best art by 5.2% mAP and 3.7% NDS on the validation set. Code will be released hrefhttps://github.com/Divadi/SOLOFusionhere.

Key facts. Cited by 177 papers in the map (data through 7 Oct 2026). Builds on 52 of its 59 references that are in the map. First posted on arXiv 5 Oct 2022.

Builds on

52 of 59 references.

References · 52

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  3. 2022A ConvNet for the 2020scited by 3,545
  4. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  5. 2019Objects as Pointscited by 1,172
  6. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  7. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  8. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
  9. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 819
  10. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 780
  11. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 721
  12. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 711
38 more
  1. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 655
  2. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 568
  3. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 550
  4. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 455
  5. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 444
  6. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 437
  7. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 351
  8. 2019Class-balanced Grouping and Sampling for Point Cloud 3D Object Detectioncited by 350
  9. 2022BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detectioncited by 340
  10. 2021Categorical Depth Distribution Network for Monocular 3D Object Detectioncited by 326
  11. 20163D Bounding Box Estimation Using Deep Learning and Geometrycited by 277
  12. 2019Recurrent MVSNet for High-resolution Multi-view Stereo Depth Inferencecited by 252
  13. 2019M3D-RPN: Monocular 3D Region Proposal Network for Object Detectioncited by 251
  14. 2021NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Videocited by 241
  15. 2019Group-wise Correlation Stereo Networkcited by 235
  16. 2019Disentangling Monocular 3D Object Detectioncited by 215
  17. 2022Unifying Voxel-based Representation with Transformer for 3D Object Detectioncited by 202
  18. 2018Orthographic Feature Transform for Monocular 3D Object Detectioncited by 201
  19. 2017Learning a Multi-View Stereo Machinecited by 194
  20. 2021Probabilistic and Geometric Depth: Detecting Objects in Perspectivecited by 190
  21. 2021Objects are Different: Flexible Monocular 3D Object Detectioncited by 184
  22. 2021ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detectioncited by 157
  23. 2019Cost Volume Pyramid Based Depth Inference for Multi-View Stereocited by 154
  24. 2015Stereo Matching by Training a Convolutional Neural Network to Compare Image Patchescited by 151
  25. 2022PolarFormer: Multi-camera 3D Object Detection with Polar Transformercited by 147
  26. 2017SurfaceNet: An End-to-end 3D Neural Network for Multiview Stereopsiscited by 128
  27. 2018ROI-10D: Monocular Lifting of 2D Detection to 6D Pose and Metric Shapecited by 123
  28. 2019Monocular 3D Object Detection with Pseudo-LiDAR Point Cloudcited by 113
  29. 2022BEVStereo: Enhancing Depth Estimation in Multi-view 3D Object Detection with Dynamic Temporal Stereocited by 89
  30. 2021AutoShape: Real-Time Shape-Aware Monocular 3D Object Detectioncited by 87
  31. 2018MonoGRNet: A Geometric Reasoning Network for Monocular 3D Object Localizationcited by 84
  32. 2022Polar Parametrization for Vision-based Surround-View 3D Detectioncited by 53
  33. 2022STS: Surround-view Temporal Stereo for Multi-view 3D Detectioncited by 53
  34. 2019Multi-View Stereo by Temporal Nonparametric Fusioncited by 46
  35. 2021Multi-View Depth Estimation by Fusing Single-View Depth Probability with Multi-View Geometrycited by 46
  36. 2022Diversity Matters: Fully Exploiting Depth Clues for Reliable Monocular 3D Object Detectioncited by 45
  37. 2022Monocular 3D Object Detection with Depth from Motioncited by 38
  38. 2019Monocular 3D Object Detection via Geometric Reasoning on Keypointscited by 31

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 48; 2024: 64; 2025: 42; 2026: 17.

177 papers.

Sort:

By area:

  1. 2022Planning-oriented Autonomous Drivingcited by 1,068
  2. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 325
  3. 2023Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detectioncited by 313
  4. 2023Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?cited by 242
  5. 2023Panacea: Panoramic and Controllable Video Generation for Autonomous Drivingcited by 165
  6. 2023FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformationcited by 162
  7. 2022Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusioncited by 161
  8. 2024Enhancing End-to-End Autonomous Driving with Latent World Modelcited by 161
  9. 2024GaussianFormer: Scene as Gaussians for Vision-Based 3D Semantic Occupancy Predictioncited by 155
  10. 2023SparseBEV: High-Performance Sparse 3D Object Detection from Multi-Camera Videoscited by 149
  11. 2023StreamMapNet: Streaming Mapping Network for Vectorized Online HD Map Constructioncited by 143
  12. 2025OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoningcited by 142
38 more
  1. 2023RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervisioncited by 126
  2. 2025DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Drivingcited by 125
  3. 2023PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentationcited by 114
  4. 2023FB-BEV: BEV Representation from Forward-Backward View Transformationscited by 98
  5. 2023Sparse4D v2: Recurrent Temporal Fusion with Sparse Modelcited by 92
  6. 2023SparseFusion: Fusing Multi-Modal Sparse Representations for Multi-Sensor 3D Object Detectioncited by 83
  7. 2023CRN: Camera Radar Net for Accurate, Robust, Efficient 3D Perceptioncited by 81
  8. 2022BEVPoolv2: A Cutting-edge Implementation of BEVDet Toward Deploymentcited by 79
  9. 2023BEVHeight: A Robust Framework for Vision-based Roadside 3D Object Detectioncited by 74
  10. 2023Far3D: Expanding the Horizon for Surround-view 3D Object Detectioncited by 70
  11. 2023Sparse4D v3: Advancing End-to-End 3D Detection and Trackingcited by 69
  12. 2023FusionAD: Multi-modality Fusion for Prediction and Planning Tasks of Autonomous Drivingcited by 67
  13. 2023Exploring Recurrent Long-term Temporal Fusion for Multi-view 3D Perceptioncited by 65
  14. 2025HERMES: A Unified Self-Driving World Model for Simultaneous 3D Scene Understanding and Generationcited by 56
  15. 2023NeRF-Det: Learning Geometry-Aware Volumetric Representation for Multi-View 3D Object Detectioncited by 51
  16. 2023Temporal Enhanced Training of Multi-view 3D Object Detector via Historical Object Predictioncited by 49
  17. 2023OctreeOcc: Efficient and Multi-Granularity Occupancy Prediction Using Octree Queriescited by 48
  18. 2024SparseAD: Sparse Query-Centric Paradigm for Efficient End-to-End Autonomous Drivingcited by 46
  19. 2023RoboBEV: Towards Robust Bird's Eye View Perception under Corruptionscited by 42
  20. 2024Benchmarking and Improving Bird's Eye View Perception Robustness in Autonomous Drivingcited by 39
  21. 2024A Survey on Occupancy Perception for Autonomous Driving: The Information Fusion Perspectivecited by 37
  22. 2024Ray Denoising: Depth-aware Hard Negative Sampling for Multi-view 3D Object Detectioncited by 36
  23. 2024ViewFormer: Exploring Spatiotemporal Modeling for Multi-View 3D Occupancy Perception via View-Guided Transformerscited by 36
  24. 2023BEVNeXt: Reviving Dense BEV Frameworks for 3D Object Detectioncited by 35
  25. 2024Unleashing HyDRa: Hybrid Fusion, Depth Consistency and Radar for Unified 3D Perceptioncited by 35
  26. 2024GraphBEV: Towards Robust BEV Feature Alignment for Multi-Modal 3D Object Detectioncited by 33
  27. 2024ADA-Track++: End-to-End Multi-Camera 3D Multi-Object Tracking with Alternating Detection and Associationcited by 33
  28. 2023UniWorld: Autonomous Driving Pre-training via World Modelscited by 32
  29. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 32
  30. 2023BEVHeight++: Toward Robust Visual Centric 3D Object Detectioncited by 29
  31. 2023X3KD: Knowledge Distillation Across Modalities, Tasks and Stages for Multi-Camera 3D Object Detectioncited by 28
  32. 2023Detecting As Labeling: Rethinking LiDAR-camera Fusion in 3D Object Detectioncited by 28
  33. 20233DiffTection: 3D Object Detection with Geometry-Aware Diffusion Featurescited by 26
  34. 2024OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoningcited by 26
  35. 2023Grid-Centric Traffic Scenario Perception for Autonomous Driving: A Comprehensive Reviewcited by 25
  36. 2024SparseLIF: High-Performance Sparse LiDAR-Camera Fusion for 3D Object Detectioncited by 25
  37. 2022Are We Ready for Vision-Centric Driving Streaming Perception? The ASAP Benchmarkcited by 24
  38. 2023PointBeV: A Sparse Approach to BeV Predictionscited by 23

50 most cited of 177 All 177 (JSON)