mlsanityData to 7 Oct 2026

Exploring Recurrent Long-term Temporal Fusion for Multi-view 3D Perception

arXiv 2303.05970 ↗First posted 10 Mar 2023cs.CVcited by 65

Not on the map. Why?

Abstract (arXiv)

Long-term temporal fusion is a crucial but often overlooked technique in camera-based Bird's-Eye-View (BEV) 3D perception. Existing methods are mostly in a parallel manner. While parallel fusion can benefit from long-term information, it suffers from increasing computational and memory overheads as the fusion window size grows. Alternatively, BEVFormer adopts a recurrent fusion pipeline so that history information can be efficiently integrated, yet it fails to benefit from longer temporal frames. In this paper, we explore an embarrassingly simple long-term recurrent fusion strategy built upon the LSS-based methods and find it already able to enjoy the merits from both sides, i.e., rich long-term information and efficient fusion pipeline. A temporal embedding module is further proposed to improve the model's robustness against occasionally missed frames in practical scenarios. We name this simple but effective fusing pipeline VideoBEV. Experimental results on the nuScenes benchmark show that VideoBEV obtains strong performance on various camera-based 3D perception tasks, including object detection (55.4% mAP and 62.9% NDS), segmentation (48.6% vehicle mIoU), tracking (54.8% AMOTA), and motion prediction (0.80m minADE and 0.463 EPA).

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). Builds on 61 of its 71 references that are in the map. First posted on arXiv 10 Mar 2023.

Builds on

61 of 71 references.

References · 61

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  5. 2022A ConvNet for the 2020scited by 3,545
  6. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  7. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  8. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  9. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,573
  10. 2015Spatial Transformer Networkscited by 1,423
  11. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
  12. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
38 more
  1. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  2. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,067
  3. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  4. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 721
  5. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 711
  6. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 655
  7. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 568
  8. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 550
  9. 2020Tracking Objects as Pointscited by 483
  10. 2022InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutionscited by 479
  11. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 444
  12. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 351
  13. 2022BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detectioncited by 340
  14. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
  15. 20163D Bounding Box Estimation Using Deep Learning and Geometrycited by 277
  16. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  17. 2020Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Netcited by 228
  18. 2022Unifying Voxel-based Representation with Transformer for 3D Object Detectioncited by 202
  19. 2022Time Will Tell: New Outlooks and A Baseline for Temporal Multi-View 3D Object Detectioncited by 177
  20. 2019Learning Depth-Guided Convolutions for Monocular 3D Object Detectioncited by 169
  21. 2022Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusioncited by 161
  22. 2022M2BEV: Multi-Camera Joint 3D Detection and Segmentation with Unified Birds-Eye View Representationcited by 155
  23. 2022PolarFormer: Multi-camera 3D Object Detection with Polar Transformercited by 147
  24. 2021Offboard 3D Object Detection from Point Cloud Sequencescited by 144
  25. 2020PnPNet: End-to-End Perception and Prediction with Tracking in the Loopcited by 143
  26. 2022ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queriescited by 116
  27. 2021RSN: Range Sparse Net for Efficient, Accurate LiDAR 3D Object Detectioncited by 115
  28. 2021Translating Images into Mapscited by 115
  29. 2022MUTR3D: A Multi-camera Tracking Framework via 3D-to-2D Queriescited by 90
  30. 2022BEVStereo: Enhancing Depth Estimation in Multi-view 3D Object Detection with Dynamic Temporal Stereocited by 89
  31. 2021AFDetV2: Rethinking the Necessity of the Second Stage for Object Detection from Point Cloudscited by 88
  32. 2020PiP: Planning-informed Trajectory Prediction for Autonomous Drivingcited by 80
  33. 2021Monocular Quasi-Dense 3D Object Trackingcited by 80
  34. 20213D-MAN: 3D Multi-frame Attention Network for Object Detectioncited by 73
  35. 2022MPPNet: Multi-Frame Feature Intertwining with Proxy Points for 3D Temporal Object Detectioncited by 65
  36. 2021DEFT: Detection Embeddings for Trackingcited by 56
  37. 2023Standing Between Past and Future: Spatio-Temporal Modeling for Multi-Camera 3D Multi-Object Trackingcited by 54
  38. 2022Polar Parametrization for Vision-based Surround-View 3D Detectioncited by 53

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 12; 2024: 30; 2025: 16; 2026: 7.

65 papers.

Sort:

By area:

  1. 2023StreamMapNet: Streaming Mapping Network for Vectorized Online HD Map Constructioncited by 143
  2. 2024DreamBench++: A Human-Aligned Benchmark for Personalized Image Generationcited by 131
  3. 2023Sparse4D v2: Recurrent Temporal Fusion with Sparse Modelcited by 92
  4. 2023Sparse4D v3: Advancing End-to-End 3D Detection and Trackingcited by 69
  5. 2024MapTracker: Tracking with Strided Memory Fusion for Consistent Vector HD Mappingcited by 67
  6. 2024HIMap: HybrId Representation Learning for End-to-end Vectorized HD Map Constructioncited by 48
  7. 2024SparseAD: Sparse Query-Centric Paradigm for Efficient End-to-End Autonomous Drivingcited by 46
  8. 2024Stream Query Denoising for Vectorized HD Map Constructioncited by 43
  9. 2024Benchmarking and Improving Bird's Eye View Perception Robustness in Autonomous Drivingcited by 39
  10. 2023BEVNeXt: Reviving Dense BEV Frameworks for 3D Object Detectioncited by 35
  11. 2023UniWorld: Autonomous Driving Pre-training via World Modelscited by 32
  12. 2024SparseLIF: High-Performance Sparse LiDAR-Camera Fusion for 3D Object Detectioncited by 25
38 more
  1. 2023PointBeV: A Sparse Approach to BeV Predictionscited by 23
  2. 2024RaCFormer: Towards High-Quality 3D Object Detection via Query-based Radar-Camera Fusioncited by 19
  3. 2023MachMap: End-to-End Vectorized Solution for Compact HD-Map Constructioncited by 17
  4. 2023UniScene: Multi-Camera Unified Pre-training via 3D Scene Reconstruction for Autonomous Drivingcited by 15
  5. 2023Leveraging Vision-Centric Multi-Modal Expertise for 3D Object Detectioncited by 13
  6. 2024PrevPredMap: Exploring Temporal Modeling with Previous Predictions for Online Vectorized HD Map Constructioncited by 12
  7. 2023OCBEV: Object-Centric BEV Transformer for Multi-View 3D Object Detectioncited by 10
  8. 2024GeoBEV: Learning Geometric BEV Representation for Multi-view 3D Object Detectioncited by 10
  9. 2024SimPB: A Single Model for 2D and 3D Object Detection from Multiple Camerascited by 9
  10. 2024MemFusionMap: Working Memory Fusion for Online Vectorized HD Map Constructioncited by 9
  11. 2025Rethinking Temporal Fusion with a Unified Gradient Descent View for 3D Semantic Occupancy Predictioncited by 9
  12. 2025Bridging Perspectives: Foundation Model Guided BEV Maps for 3D Object Detection and Trackingcited by 9
  13. 2023Lift-Attend-Splat: Bird's-eye-view camera-lidar fusion using transformerscited by 8
  14. 2024MIM4D: Masked Modeling with Multi-View Video for Autonomous Driving Representation Learningcited by 8
  15. 2024SparseFusion: Efficient Sparse Multi-Modal Fusion Framework for Long-Range 3D Perceptioncited by 8
  16. 2025HisTrackMap: Global Vectorized High-Definition Map Construction via History Map Trackingcited by 8
  17. 2025MamBEV: Enabling State Space Models to Learn Birds-Eye-View Representationscited by 8
  18. 2025InteractionMap: Improving Online Vectorized HDMap Construction with Interactioncited by 8
  19. 2024UniVision: A Unified Framework for Vision-Centric 3D Perceptioncited by 7
  20. 2025AugMapNet: Improving Spatial Latent Structure via BEV Grid Augmentation for Enhanced Vectorized Online HD Map Constructioncited by 6
  21. 2023ClusterFusion: Leveraging Radar Spatial Features for Radar-Camera 3D Object Detection in Autonomous Vehiclescited by 5
  22. 2024TempBEV: Improving Learned BEV Encoders with Combined Image and BEV Space Temporal Aggregationcited by 5
  23. 2024MambaBEV: An EV-based 3D detection model with Mamba2cited by 4
  24. 2024PolarBEVDet: Exploring Polar Representation for Multi-View 3D Object Detection in Bird's-Eye-Viewcited by 3
  25. 2024EgoFSD: Ego-Centric Fully Sparse Paradigm with Uncertainty Denoising and Iterative Refinement for Efficient End-to-End Self-Drivingcited by 3
  26. 2024HV-BEV: Decoupling Horizontal and Vertical Feature Sampling for Multi-View 3D Object Detectioncited by 3
  27. 2025Coherent Online Road Topology Estimation and Reasoning with Standard-Definition Mapscited by 3
  28. 2025MambaFusion: Height-Fidelity Dense Global Fusion for Multi-modal 3D Object Detectioncited by 3
  29. 2025ForeSight: Multi-View Streaming Joint Object Detection and Trajectory Forecastingcited by 3
  30. 2025Mapping like a Skeptic: Probabilistic BEV Projection for Online HD Mappingcited by 3
  31. 2025UniLION: Towards Unified Autonomous Driving Model with Linear Group RNNscited by 3
  32. 2024DuoSpaceNet: Leveraging Both Bird's-Eye-View and Perspective View Representations for 3D Object Detectioncited by 2
  33. 2024LSSInst: Improving Geometric Modeling in LSS-Based BEV Perception with Instance Representationcited by 2
  34. 2026Leveraging Previous-Traversal Point Cloud Map Priors for Camera-Based 3D Object Detection and Trackingcited by 2
  35. 2024Learning Temporal Cues by Predicting Objects Move for Multi-camera 3D Object Detectioncited by 1
  36. 2024Learning Shared RGB-D Fields: Unified Self-supervised Pre-training for Label-efficient LiDAR-Camera 3D Perceptioncited by 1
  37. 2024DVPE: Divided View Position Embedding for Multi-View 3D Object Detectioncited by 1
  38. 2024Epipolar Attention Field Transformers for Bird's Eye View Semantic Segmentationcited by 1

50 most cited of 65 All 65 (JSON)