mlsanityData to 14 Oct 2026

Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusion

arXiv 2211.10581 ↗First posted 19 Nov 2022cs.CVcited by 162

Not on the map. Why?

Cited by 26 in End-to-end autonomous driving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Bird-eye-view (BEV) based methods have made great progress recently in multi-view 3D detection task. Comparing with BEV based methods, sparse based methods lag behind in performance, but still have lots of non-negligible merits. To push sparse 3D detection further, in this work, we introduce a novel method, named Sparse4D, which does the iterative refinement of anchor boxes via sparsely sampling and fusing spatial-temporal features. (1) Sparse 4D Sampling: for each 3D anchor, we assign multiple 4D keypoints, which are then projected to multi-view/scale/timestamp image features to sample corresponding features; (2) Hierarchy Feature Fusion: we hierarchically fuse sampled features of different view/scale, different timestamp and different keypoints to generate high-quality instance feature. In this way, Sparse4D can efficiently and effectively achieve 3D detection without relying on dense view transformation nor global attention, and is more friendly to edge devices deployment. Furthermore, we introduce an instance-level depth reweight module to alleviate the ill-posed issue in 3D-to-2D projection. In experiment, our method outperforms all sparse based methods and most BEV based methods on detection task in the nuScenes dataset.

Key facts. Cited by 162 papers in the map (data through 14 Oct 2026). Builds on 46 of its 52 references that are in the map. First posted on arXiv 19 Nov 2022.

Builds on

46 of 52 references.

References · 46

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  3. 2020End-to-End Object Detection with Transformerscited by 7,414
  4. 2017Mask R-CNNcited by 7,277
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  6. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  7. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  8. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  9. 2015SSD: Single Shot MultiBox Detectorcited by 3,130
  10. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,577
  11. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  12. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,044
34 more
  1. 2019EfficientDet: Scalable and Efficient Object Detectioncited by 1,020
  2. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 788
  3. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 713
  4. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 658
  5. 2019CenterNet: Keypoint Triplets for Object Detectioncited by 654
  6. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 572
  7. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 552
  8. 2020Sparse R-CNN: End-to-End Object Detection with Learnable Proposalscited by 495
  9. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 457
  10. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 447
  11. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 352
  12. 2019Class-balanced Grouping and Sampling for Point Cloud 3D Object Detectioncited by 350
  13. 2022BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detectioncited by 341
  14. 2022BEVFusion: A Simple and Robust LiDAR-Camera Fusion Frameworkcited by 336
  15. 2021Categorical Depth Distribution Network for Monocular 3D Object Detectioncited by 327
  16. 2021Is Pseudo-Lidar needed for Monocular 3D Object detection?cited by 249
  17. 2020SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimationcited by 204
  18. 2018Orthographic Feature Transform for Monocular 3D Object Detectioncited by 202
  19. 2022Unifying Voxel-based Representation with Transformer for 3D Object Detectioncited by 202
  20. 2022BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Drivingcited by 191
  21. 2022Time Will Tell: New Outlooks and A Baseline for Temporal Multi-View 3D Object Detectioncited by 178
  22. 2019An Energy and GPU-Computation Efficient Backbone Network for Real-Time Object Detectioncited by 163
  23. 2019Monocular 3D Object Detection with Pseudo-LiDAR Point Cloudcited by 113
  24. 2019Deep Learning in Video Multi-Object Tracking: A Surveycited by 107
  25. 2022MUTR3D: A Multi-camera Tracking Framework via 3D-to-2D Queriescited by 91
  26. 2022BEVStereo: Enhancing Depth Estimation in Multi-view 3D Object Detection with Dynamic Temporal Stereocited by 90
  27. 2020End-to-End Pseudo-LiDAR for Image-Based 3D Object Detectioncited by 80
  28. 2019CAM-Convs: Camera-Aware Multi-Scale Convolutions for Single-View Depthcited by 75
  29. 2022BEVDistill: Cross-Modal BEV Distillation for Multi-View 3D Object Detectioncited by 66
  30. 2022Polar Parametrization for Vision-based Surround-View 3D Detectioncited by 53
  31. 20193D Multi-Object Tracking: A Baseline and New Evaluation Metricscited by 47
  32. 2022Graph-DETR3D: Rethinking Overlapping Regions for Multi-View 3D Object Detectioncited by 35
  33. 2022Quality Matters: Embracing Quality Clues for Robust 3D Multi-Object Trackingcited by 17
  34. 2022A Simple Baseline for Multi-Camera 3D Object Detectioncited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 25; 2024: 51; 2025: 47; 2026: 39.

162 papers.

Sort:

By area:

  1. 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 334
  2. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 332
  3. 2023Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detectioncited by 315
  4. 2024SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representationcited by 241
  5. 2023MapTRv2: An End-to-End Framework for Online Vectorized HD Map Constructioncited by 193
  6. 2023SparseBEV: High-Performance Sparse 3D Object Detection from Multi-Camera Videoscited by 150
  7. 2023RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervisioncited by 128
  8. 2023FB-BEV: BEV Representation from Forward-Backward View Transformationscited by 98
  9. 2025World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Modelcited by 95
  10. 2023Sparse4D v2: Recurrent Temporal Fusion with Sparse Modelcited by 93
  11. 2023CRN: Camera Radar Net for Accurate, Robust, Efficient 3D Perceptioncited by 82
  12. 2023Far3D: Expanding the Horizon for Surround-view 3D Object Detectioncited by 70
38 more
  1. 2023Sparse4D v3: Advancing End-to-End 3D Detection and Trackingcited by 70
  2. 2024GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Predictioncited by 67
  3. 2023Exploring Recurrent Long-term Temporal Fusion for Multi-view 3D Perceptioncited by 65
  4. 2023Fully Sparse 3D Occupancy Predictioncited by 64
  5. 2024OPUS: Occupancy Prediction Using a Sparse Setcited by 56
  6. 2023Temporal Enhanced Training of Multi-view 3D Object Detector via Historical Object Predictioncited by 49
  7. 2023OctreeOcc: Efficient and Multi-Granularity Occupancy Prediction Using Octree Queriescited by 48
  8. 2024SparseAD: Sparse Query-Centric Paradigm for Efficient End-to-End Autonomous Drivingcited by 47
  9. 2024Navigation-Guided Sparse Scene Representation for End-to-End Autonomous Drivingcited by 46
  10. 2023RoboBEV: Towards Robust Bird's Eye View Perception under Corruptionscited by 42
  11. 2024Benchmarking and Improving Bird's Eye View Perception Robustness in Autonomous Drivingcited by 41
  12. 2024Ray Denoising: Depth-aware Hard Negative Sampling for Multi-view 3D Object Detectioncited by 36
  13. 2023BEVNeXt: Reviving Dense BEV Frameworks for 3D Object Detectioncited by 35
  14. 2025HiP-AD: Hierarchical and Multi-Granularity Planning with Deformable Attention for Autonomous Driving in a Single Decodercited by 35
  15. 2024ADA-Track++: End-to-End Multi-Camera 3D Multi-Object Tracking with Alternating Detection and Associationcited by 33
  16. 2025Detect Anything 3D in the Wildcited by 33
  17. 2024Hierarchical Temporal Context Learning for Camera-based Semantic Scene Completioncited by 32
  18. 2026SparseDriveV2: Scoring is All You Need for End-to-End Autonomous Drivingcited by 26
  19. 2024SparseLIF: High-Performance Sparse LiDAR-Camera Fusion for 3D Object Detectioncited by 25
  20. 2024The RoboDrive Challenge: Drive Anytime Anywhere in Any Conditioncited by 23
  21. 2024MV2DFusion: Leveraging Modality-Specific Object Semantics for Multi-Modal 3D Detectioncited by 23
  22. 2025STCOcc: Sparse Spatial-Temporal Cascade Renovation for 3D Occupancy and Scene Flow Predictioncited by 23
  23. 2025Doracamom: Joint 3D Detection and Occupancy Prediction with Multi-view 4D Radars and Cameras for Omnidirectional Perceptioncited by 22
  24. 2024Is a 3D-Tokenized LLM the Key to Reliable Autonomous Driving?cited by 20
  25. 2023HeightFormer: Explicit Height Modeling without Extra Data for Camera-only 3D Object Detection in Bird's Eye Viewcited by 19
  26. 2023M-BEV: Masked BEV Perception for Robust Autonomous Drivingcited by 19
  27. 2024Neural Radiance Field in Autonomous Driving: A Surveycited by 19
  28. 2024RaCFormer: Towards High-Quality 3D Object Detection via Query-based Radar-Camera Fusioncited by 19
  29. 2026UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Drivingcited by 19
  30. 2025GaussRender: Learning 3D Occupancy with Gaussian Renderingcited by 18
  31. 2024HENet: Hybrid Encoding for End-to-end Multi-task 3D Perception from Multi-view Camerascited by 16
  32. 2023FusionFormer: A Multi-sensory Fusion in Bird's-Eye-View and Temporal Consistent Transformer for 3D Object Detectioncited by 14
  33. 2024CLIP-BEVFormer: Enhancing Multi-View Image-Based BEV Detector with Ground Truth Flowcited by 14
  34. 2025DistillDrive: End-to-End Multi-Mode Autonomous Driving Distillation by Isomorphic Hetero-Source Planning Modelcited by 14
  35. 2025CoCMT: Communication-Efficient Cross-Modal Transformer for Collaborative Perceptioncited by 13
  36. 2026DVGT-2: Vision-Geometry-Action Model for Autonomous Driving at Scalecited by 13
  37. 2024PrevPredMap: Exploring Temporal Modeling with Previous Predictions for Online Vectorized HD Map Constructioncited by 12
  38. 2024GeoBEV: Learning Geometric BEV Representation for Multi-view 3D Object Detectioncited by 11

50 most cited of 162 All 162 (JSON)