# DiffBEV: Conditional Diffusion Model for Bird's Eye View Perception

arXiv: https://arxiv.org/abs/2303.08333 · HTML: https://mlsanity.com/paper/2303.08333/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 47 of its 53 references that are in the map.
- First posted on arXiv 15 Mar 2023.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 47)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Make-A-Video: Text-to-Video Generation without Text-Video Data](https://mlsanity.com/paper/2209.14792/) (2022)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [Planning with Diffusion for Flexible Behavior Synthesis](https://mlsanity.com/paper/2205.09991/) (2022)
- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)
- [Vector Quantized Diffusion Model for Text-to-Image Synthesis](https://mlsanity.com/paper/2111.14822/) (2021)
- [Diffusion Probabilistic Models for 3D Point Cloud Generation](https://mlsanity.com/paper/2103.01458/) (2021)
- [BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View](https://mlsanity.com/paper/2112.11790/) (2021)
- [DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries](https://mlsanity.com/paper/2110.06922/) (2021)
- [CenterNet: Keypoint Triplets for Object Detection](https://mlsanity.com/paper/1904.08189/) (2019)
- [DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation](https://mlsanity.com/paper/2110.02711/) (2021)
- [ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2108.02938/) (2021)
- [BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection](https://mlsanity.com/paper/2206.10092/) (2022)
- [LION: Latent Point Diffusion Models for 3D Shape Generation](https://mlsanity.com/paper/2210.06978/) (2022)
- [PETR: Position Embedding Transformation for Multi-View 3D Object Detection](https://mlsanity.com/paper/2203.05625/) (2022)
- [3D Shape Generation and Completion through Point-Voxel Diffusion](https://mlsanity.com/paper/2104.03670/) (2021)
- [Tackling the Generative Learning Trilemma with Denoising Diffusion GANs](https://mlsanity.com/paper/2112.07804/) (2021)
- [Label-Efficient Semantic Segmentation with Diffusion Models](https://mlsanity.com/paper/2112.03126/) (2021)
- [FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection](https://mlsanity.com/paper/2104.10956/) (2021)
- [Blended Latent Diffusion](https://mlsanity.com/paper/2206.02779/) (2022)
- [DiffusionDet: Diffusion Model for Object Detection](https://mlsanity.com/paper/2211.09788/) (2022)
- [BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detection](https://mlsanity.com/paper/2203.17054/) (2022)

## Built on by (most cited first, 30 of 30)

- [DiffAD: A Unified Diffusion Modeling Approach for Autonomous Driving](https://mlsanity.com/paper/2503.12170/) (2025)
- [DifFUSER: Diffusion Model for Robust Multi-Sensor Fusion in 3D Object Detection and BEV Segmentation](https://mlsanity.com/paper/2404.04629/) (2024)
- [BEVDiffuser: Plug-and-Play Diffusion Model for BEV Denoising with Ground-Truth Guidance](https://mlsanity.com/paper/2502.19694/) (2025)
- [DIVER: Reinforced Diffusion Breaks Imitation Bottlenecks in End-to-End Autonomous Driving](https://mlsanity.com/paper/2507.04049/) (2025)
- [Improving Bird's Eye View Semantic Segmentation by Task Decomposition](https://mlsanity.com/paper/2404.01925/) (2024)
- [MapDiffusion: Generative Diffusion for Vectorized Online HD Map Construction and Uncertainty Estimation in Autonomous Driving](https://mlsanity.com/paper/2507.21423/) (2025)
- [BLOS-BEV: Navigation Map Enhanced Lane Segmentation Network, Beyond Line of Sight](https://mlsanity.com/paper/2407.08526/) (2024)
- [Pragmatic Heterogeneous Collaborative Perception via Generative Communication Mechanism](https://mlsanity.com/paper/2510.19618/) (2025)
- [DiffCP: Ultra-Low Bit Collaborative Perception via Diffusion Model](https://mlsanity.com/paper/2409.19592/) (2024)
- [DualDiff: Dual-branch Diffusion Model for Autonomous Driving with Semantic Fusion](https://mlsanity.com/paper/2505.01857/) (2025)
- [GUPNet++: Geometry Uncertainty Propagation Network for Monocular 3D Object Detection](https://mlsanity.com/paper/2310.15624/) (2023)
- [Intriguing Properties of Diffusion Models: An Empirical Study of the Natural Attack Capability in Text-to-Image Generative Models](https://mlsanity.com/paper/2308.15692/) (2023)
- [CoDiff: Conditional Diffusion Model for Collaborative 3D Object Detection](https://mlsanity.com/paper/2502.14891/) (2025)
- [DiffRef3D: A Diffusion-based Proposal Refinement Framework for 3D Object Detection](https://mlsanity.com/paper/2310.16349/) (2023)
- [Diffusion-Based Particle-DETR for BEV Perception](https://mlsanity.com/paper/2312.11578/) (2023)
- [Focus on BEV: Self-calibrated Cycle View Transformation for Monocular Birds-Eye-View Segmentation](https://mlsanity.com/paper/2410.15932/) (2024)
- [VQ-Map: Bird's-Eye-View Map Layout Estimation in Tokenized Discrete Space via Vector Quantization](https://mlsanity.com/paper/2411.01618/) (2024)
- [RESAR-BEV: An Explainable Progressive Residual Autoregressive Approach for Camera-Radar Fusion in BEV Segmentation](https://mlsanity.com/paper/2505.06515/) (2025)
- [Diffusion-Based Generative Models for 3D Occupancy Prediction in Autonomous Driving](https://mlsanity.com/paper/2505.23115/) (2025)
- [EffiPerception: an Efficient Framework for Various Perception Tasks](https://mlsanity.com/paper/2403.12317/) (2024)
- [RESBev: Making BEV Perception More Robust](https://mlsanity.com/paper/2603.09529/) (2026)
- [Reconstruction Matters: Learning Geometry-Aligned BEV Representation through 3D Gaussian Splatting](https://mlsanity.com/paper/2603.19193/) (2026)
- [Learning Ego-Centric BEV Representations from a Perspective-Privileged View: Cross-View Supervision for Online HD Map Construction](https://mlsanity.com/paper/2605.12218/) (2026)
- [FMOcc: TPV-Driven Flow Matching for 3D Occupancy Prediction with Selective State Space Model](https://mlsanity.com/paper/2507.02250/) (2025)
- [DRCP: Diffusion on Reinforced Cooperative Perception for Perceiving Beyond Limits](https://mlsanity.com/paper/2509.24903/) (2025)
- [CycleBEV: Regularizing View Transformation Networks via View Cycle Consistency for Bird's-Eye-View Semantic Segmentation](https://mlsanity.com/paper/2602.23575/) (2026)
- [FlatLands: Generative Floormap Completion From a Single Egocentric View](https://mlsanity.com/paper/2603.16016/) (2026)
- [Generative-Model Predictive Planning for Navigation in Partially Observable Environments](https://mlsanity.com/paper/2606.18888/) (2026)
- [BEV-Denoise: Learning Intrinsic Noise for Accurate Bird's-Eye-View Semantic Segmentation](https://mlsanity.com/paper/2606.22931/) (2026)
- [Open-Vocabulary BEV Segmentation with 3D-Aware Geometric Constraints](https://mlsanity.com/paper/2606.24353/) (2026)

## Abstract (arXiv)

BEV perception is of great importance in the field of autonomous driving, serving as the cornerstone of planning, controlling, and motion prediction. The quality of the BEV feature highly affects the performance of BEV perception. However, taking the noises in camera parameters and LiDAR scans into consideration, we usually obtain BEV representation with harmful noises. Diffusion models naturally have the ability to denoise noisy samples to the ideal data, which motivates us to utilize the diffusion model to get a better BEV representation. In this work, we propose an end-to-end framework, named DiffBEV, to exploit the potential of diffusion model to generate a more comprehensive BEV representation. To the best of our knowledge, we are the first to apply diffusion model to BEV perception. In practice, we design three types of conditions to guide the training of the diffusion model which denoises the coarse samples and refines the semantic feature in a progressive way. What's more, a cross-attention module is leveraged to fuse the context of BEV feature and the semantic content of conditional diffusion model. DiffBEV achieves a 25.9% mIoU on the nuScenes dataset, which is 6.2% higher than the best-performing existing approach. Quantitative and qualitative results on multiple benchmarks demonstrate the effectiveness of DiffBEV in BEV semantic segmentation and 3D object detection tasks. The code will be available soon.