mlsanityData to 14 Oct 2026

Sparse4D v3: Advancing End-to-End 3D Detection and Tracking

arXiv 2311.11722 ↗First posted 20 Nov 2023cs.CV cs.AI cs.ROcited by 70

Not on the map. Why?

Abstract (arXiv)

In autonomous driving perception systems, 3D detection and tracking are the two fundamental tasks. This paper delves deeper into this field, building upon the Sparse4D framework. We introduce two auxiliary training tasks (Temporal Instance Denoising and Quality Estimation) and propose decoupled attention to make structural improvements, leading to significant enhancements in detection performance. Additionally, we extend the detector into a tracker using a straightforward approach that assigns instance ID during inference, further highlighting the advantages of query-based algorithms. Extensive experiments conducted on the nuScenes benchmark validate the effectiveness of the proposed improvements. With ResNet50 as the backbone, we witnessed enhancements of 3.0%, 2.2%, and 7.6% in mAP, NDS, and AMOTA, achieving 46.9%, 56.1%, and 49.0%, respectively. Our best model achieved 71.9% NDS and 67.7% AMOTA on the nuScenes test set. Code will be released at urlhttps://github.com/linxuewu/Sparse4D.

Key facts. Cited by 70 papers in the map (data through 14 Oct 2026). Builds on 47 of its 53 references that are in the map. First posted on arXiv 20 Nov 2023.

Builds on

47 of 53 references.

References · 47

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020End-to-End Object Detection with Transformerscited by 7,414
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  5. 2022A ConvNet for the 2020scited by 3,579
  6. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  7. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
  8. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  9. 2022Planning-oriented Autonomous Drivingcited by 1,087
  10. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 713
  11. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 658
  12. 2022DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETRcited by 573
35 more
  1. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 572
  2. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 552
  3. 2021TrackFormer: Multi-Object Tracking with Transformerscited by 518
  4. 2022DN-DETR: Accelerate DETR Training by Introducing Query DeNoisingcited by 488
  5. 2022InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutionscited by 482
  6. 2022TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformerscited by 466
  7. 2021Conditional DETR for Fast Training Convergencecited by 424
  8. 2021MOTR: End-to-End Multiple-Object Tracking with Transformercited by 381
  9. 2023EVA-02: A Visual Representation for Neon Genesiscited by 381
  10. 2020TransTrack: Multiple Object Tracking with Transformercited by 371
  11. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 352
  12. 2019Class-balanced Grouping and Sampling for Point Cloud 3D Object Detectioncited by 350
  13. 2022BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detectioncited by 341
  14. 2023Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detectioncited by 315
  15. 2022MapTR: Structured Modeling and Learning for Online Vectorized HD Map Constructioncited by 314
  16. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
  17. 2022DETRs with Collaborative Hybrid Assignments Trainingcited by 274
  18. 2021Is Pseudo-Lidar needed for Monocular 3D Object detection?cited by 249
  19. 2022Time Will Tell: New Outlooks and A Baseline for Temporal Multi-View 3D Object Detectioncited by 178
  20. 2019An Energy and GPU-Computation Efficient Backbone Network for Real-Time Object Detectioncited by 163
  21. 2022Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusioncited by 162
  22. 2015MOTChallenge 2015: Towards a Benchmark for Multi-Target Trackingcited by 133
  23. 2023Sparse4D v2: Recurrent Temporal Fusion with Sparse Modelcited by 93
  24. 2022MUTR3D: A Multi-camera Tracking Framework via 3D-to-2D Queriescited by 91
  25. 2022BEVStereo: Enhancing Depth Estimation in Multi-view 3D Object Detection with Dynamic Temporal Stereocited by 90
  26. 2022BEVPoolv2: A Cutting-edge Implementation of BEVDet Toward Deploymentcited by 79
  27. 2023Exploring Recurrent Long-term Temporal Fusion for Multi-view 3D Perceptioncited by 65
  28. 2022Polar Parametrization for Vision-based Surround-View 3D Detectioncited by 53
  29. 2023Temporal Enhanced Training of Multi-view 3D Object Detector via Historical Object Predictioncited by 49
  30. 2023MOTRv3: Release-Fetch Supervision for End-to-End Multi-Object Trackingcited by 42
  31. 2023Object as Query: Lifting any 2D Object Detector to 3D Detectioncited by 38
  32. 2022Graph-DETR3D: Rethinking Overlapping Regions for Multi-View 3D Object Detectioncited by 35
  33. 2022Quality Matters: Embracing Quality Clues for Robust 3D Multi-Object Trackingcited by 17
  34. 2023DORT: Modeling Dynamic Objects in Recurrent for Multi-Camera 3D Object Detection and Trackingcited by 11
  35. 2021Global Correlation Network: End-to-End Joint Multi-Object Detection and Trackingcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 23; 2025: 24; 2026: 23.

70 papers.

Sort:

By area:

  1. 2024SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representationcited by 241
  2. 2024MapTracker: Tracking with Strided Memory Fusion for Consistent Vector HD Mappingcited by 68
  3. 2025DiffVLA: Vision-Language Guided Diffusion Planning for Autonomous Drivingcited by 64
  4. 2024SparseAD: Sparse Query-Centric Paradigm for Efficient End-to-End Autonomous Drivingcited by 47
  5. 2024SparseLIF: High-Performance Sparse LiDAR-Camera Fusion for 3D Object Detectioncited by 25
  6. 2024MV2DFusion: Leveraging Modality-Specific Object Semantics for Multi-Modal 3D Detectioncited by 23
  7. 2024Neural Radiance Field in Autonomous Driving: A Surveycited by 19
  8. 2024DAOcc: 3D Object Detection Assisted Multi-Sensor Fusion for 3D Occupancy Predictioncited by 15
  9. 2024CLIP-BEVFormer: Enhancing Multi-View Image-Based BEV Detector with Ground Truth Flowcited by 14
  10. 2025CoCMT: Communication-Efficient Cross-Modal Transformer for Collaborative Perceptioncited by 13
  11. 2026DVGT-2: Vision-Geometry-Action Model for Autonomous Driving at Scalecited by 13
  12. 2024PrevPredMap: Exploring Temporal Modeling with Previous Predictions for Online Vectorized HD Map Constructioncited by 12
38 more
  1. 2025CoopTrack: Exploring End-to-End Learning for Efficient Cooperative Sequential Perceptioncited by 12
  2. 2024BEV-TSR: Text-Scene Retrieval in BEV Space for Autonomous Drivingcited by 11
  3. 20253D-MOOD: Lifting 2D to 3D for Monocular Open-Set Object Detectioncited by 11
  4. 2024SimPB: A Single Model for 2D and 3D Object Detection from Multiple Camerascited by 9
  5. 2024Make Your ViT-based Multi-view 3D Detectors Faster via Token Compressioncited by 9
  6. 2025Bridging Perspectives: Foundation Model Guided BEV Maps for 3D Object Detection and Trackingcited by 9
  7. 2026VGGT-Det: Mining VGGT Internal Priors for Sensor-Geometry-Free Multi-View Indoor 3D Object Detectioncited by 9
  8. 2024MIM4D: Masked Modeling with Multi-View Video for Autonomous Driving Representation Learningcited by 8
  9. 2024Scaling Multi-Camera 3D Object Detection through Weak-to-Strong Elicitingcited by 7
  10. 2025RoPETR: Improving Temporal Camera-Only 3D Detection by Integrating Enhanced Rotary Position Embeddingcited by 7
  11. 2025FocalAD: Local Motion Planning for End-to-End Autonomous Drivingcited by 6
  12. 2025VR-Drive: Viewpoint-Robust End-to-End Driving with Feed-Forward 3D Gaussian Splattingcited by 6
  13. 2025Divide and Merge: Motion and Semantic Learning in End-to-End Autonomous Drivingcited by 5
  14. 2025A Survey of Multi-sensor Fusion Perception for Embodied AI: Background, Methods, Challenges and Prospectscited by 5
  15. 2024Learning High-resolution Vector Representation from Multi-Camera Images for 3D Object Detectioncited by 4
  16. 2024LiDAR-based End-to-end Temporal Perception for Vehicle-Infrastructure Cooperationcited by 4
  17. 2025TQD-Track: Temporal Query Denoising for 3D Multi-Object Trackingcited by 4
  18. 2025SparseMeXT Unlocking the Potential of Sparse Representations for HD Map Constructioncited by 4
  19. 2025SQS: Enhancing Sparse Perception Models via Query-based Splatting in Autonomous Drivingcited by 4
  20. 2025Forging Spatial Intelligence: A Roadmap of Multi-Modal Data Pre-Training for Autonomous Systemscited by 4
  21. 2024Image-Guided Outdoor LiDAR Perception Quality Assessment for Autonomous Drivingcited by 3
  22. 2024EgoFSD: Ego-Centric Fully Sparse Paradigm with Uncertainty Denoising and Iterative Refinement for Efficient End-to-End Self-Drivingcited by 3
  23. 2025TopoStreamer: Temporal Lane Segment Topology Reasoning in Autonomous Drivingcited by 3
  24. 2025Coherent Online Road Topology Estimation and Reasoning with Standard-Definition Mapscited by 3
  25. 2024DuoSpaceNet: Leveraging Both Bird's-Eye-View and Perspective View Representations for 3D Object Detectioncited by 2
  26. 2024S2-Track: A Simple yet Strong Approach for End-to-End 3D Multi-Object Trackingcited by 2
  27. 2024RockTrack: A 3D Robust Multi-Camera-Ken Multi-Object Tracking Frameworkcited by 2
  28. 2025OmniScene: Attention-Augmented Multimodal 4D Scene Understanding for Autonomous Drivingcited by 2
  29. 2026A Unified 3D Object Perception Framework for Real-Time Outside-In Multi-Camera Systemscited by 2
  30. 2026Offline-Poly: A Polyhedral Framework For Offline 3D Multi-Object Trackingcited by 2
  31. 2026SCATR: Mitigating New Instance Suppression in LiDAR-based Tracking-by-Attention via Second Chance Assignment and Track Query Dropoutcited by 2
  32. 2026Leveraging Previous-Traversal Point Cloud Map Priors for Camera-Based 3D Object Detection and Trackingcited by 2
  33. 2024MDHA: Multi-Scale Deformable Transformer with Hybrid Anchors for Multi-View 3D Object Detectioncited by 1
  34. 2025Synthesizing and Identifying Noise Levels in Autonomous Vehicle Camera Radar Datasetscited by 1
  35. 2025S2GO: Streaming Sparse Gaussian Occupancy Predictioncited by 1
  36. 2025LiteFusion: Taming 3D Object Detectors from Vision-Based to Multi-Modal with Minimal Adaptationcited by 1
  37. 2026SuperOcc: Toward Cohesive Temporal Modeling for Superquadric-based 3D Occupancy Predictioncited by 1
  38. 2026MASAR: Motion-Appearance Synergy Refinement for Joint Detection and Trajectory Forecastingcited by 1

50 most cited of 70 All 70 (JSON)