mlsanityData to 14 Oct 2026

3DifFusionDet: Diffusion Model for 3D Object Detection with Robust LiDAR-Camera Fusion

arXiv 2311.03742 ↗First posted 7 Nov 2023cs.CVcited by 4

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Good 3D object detection performance from LiDAR-Camera sensors demands seamless feature alignment and fusion strategies. We propose the 3DifFusionDet framework in this paper, which structures 3D object detection as a denoising diffusion process from noisy 3D boxes to target boxes. In this framework, ground truth boxes diffuse in a random distribution for training, and the model learns to reverse the noising process. During inference, the model gradually refines a set of boxes that were generated at random to the outcomes. Under the feature align strategy, the progressive refinement method could make a significant contribution to robust LiDAR-Camera fusion. The iterative refinement process could also demonstrate great adaptability by applying the framework to various detecting circumstances where varying levels of accuracy and speed are required. Extensive experiments on KITTI, a benchmark for real-world traffic object identification, revealed that 3DifFusionDet is able to perform favorably in comparison to earlier, well-respected detectors.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 46 of its 61 references that are in the map. First posted on arXiv 7 Nov 2023.

Builds on

46 of 61 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022DiffusionDet: Diffusion Model for Object Detection17 Nov
  2. 2021Blended Diffusion for Text-driven Editing of Natural Images29 Nov

Other references · 44

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  6. 2020End-to-End Object Detection with Transformerscited by 7,414
  7. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  8. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  9. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,162
  10. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  11. 2021Image Super-Resolution via Iterative Refinementcited by 1,511
  12. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,368
32 more
  1. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,354
  2. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  3. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 941
  4. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 891
  5. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 850
  6. 2022Equivariant Diffusion for Molecule Generation in 3Dcited by 677
  7. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  8. 2021Label-Efficient Semantic Segmentation with Diffusion Modelscited by 499
  9. 2020Sparse R-CNN: End-to-End Object Detection with Learnable Proposalscited by 495
  10. 2022DN-DETR: Accelerate DETR Training by Introducing Query DeNoisingcited by 488
  11. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 485
  12. 2022TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformerscited by 466
  13. 20203DSSD: Point-based 3D Single Stage Object Detectorcited by 440
  14. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 377
  15. 2019From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Networkcited by 368
  16. 2021An End-to-End Transformer Model for 3D Object Detectioncited by 329
  17. 2017Joint 3D Proposal Generation and Object Detection from View Aggregationcited by 326
  18. 2022Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problemcited by 236
  19. 2022Diffusion models as plug-and-play priorscited by 231
  20. 2021Embracing Single Stride 3D Object Detector with Sparse Transformercited by 193
  21. 2021OTA: Optimal Transport Assignment for Object Detectioncited by 178
  22. 2021Fast Convergence of DETR with Spatially Modulated Co-Attentioncited by 176
  23. 2019MVX-Net: Multimodal VoxelNet for 3D Object Detectioncited by 163
  24. 20223D Object Detection for Autonomous Driving: A Comprehensive Surveycited by 132
  25. 2021Improving 3D Object Detection with Channel-wise Transformercited by 108
  26. 2022Guided Diffusion Model for Adversarial Purificationcited by 92
  27. 2022SWFormer: Sparse Window Transformer for 3D Object Detection in Point Cloudscited by 92
  28. 2023Diffusion-Based 3D Human Pose Estimation with Multi-Hypothesis Aggregationcited by 82
  29. 20163D Object Proposals using Stereo Imagery for Accurate Object Class Detectioncited by 58
  30. 2023DiffusionRet: Generative Text-Video Retrieval with Diffusion Modelcited by 52
  31. 2022DiffusionInst: Diffusion Model for Instance Segmentationcited by 49
  32. 2015Multi-View Factorization Machinescited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 1.

4 papers.

  1. 2025Towards Open-Vocabulary Multimodal 3D Object Detection with Attributescited by 2
  2. 2024EffiPerception: an Efficient Framework for Various Perception Taskscited by 1
  3. 2025REXO: Indoor Multi-View Radar Object Detection via 3D Bounding Box Diffusioncited by 1
  4. 2026OccFace: Unified Occlusion-Aware Facial Landmark Detection with Per-Point Visibilitycited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2023DiffRef3D: A Diffusion-based Proposal Refinement Framework for 3D Object Detection
  2. 2024DiffuBox: Refining 3D Object Detection with Point Diffusion
  3. 2024V2X-R: Cooperative LiDAR-4D Radar Fusion with Denoising Diffusion for 3D Object Detection
  4. 2024FSMDet: Vision-guided feature diffusion for fully sparse 3D detector
  5. 2024Scaling Diffusion Models to Real-World 3D LiDAR Scene Completion
  6. 2023Diffusion-based 3D Object Detection with Random Boxes
  7. 2023SVDM: Single-View Diffusion Model for Pseudo-Stereo 3D Object Detection
  8. 2022Li3DeTr: A LiDAR based 3D Detection Transformer