# Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformers

arXiv: https://arxiv.org/abs/2510.07316 · HTML: https://mlsanity.com/paper/2510.07316/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 84 of its 98 references that are in the map.
- First posted on arXiv 8 Oct 2025.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 30 of 84)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [Building Normalizing Flows with Stochastic Interpolants](https://mlsanity.com/paper/2209.15571/) (2022)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think](https://mlsanity.com/paper/2410.06940/) (2024)
- [ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth](https://mlsanity.com/paper/2302.12288/) (2023)
- [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020)
- [TartanAir: A Dataset to Push the Limits of Visual SLAM](https://mlsanity.com/paper/2003.14338/) (2020)
- [BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networks](https://mlsanity.com/paper/1911.10127/) (2019)
- [Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture](https://mlsanity.com/paper/1411.4734/) (2014)
- [Simple diffusion: End-to-end diffusion for high resolution images](https://mlsanity.com/paper/2301.11093/) (2023)
- [Depth Pro: Sharp Monocular Metric Depth in Less Than a Second](https://mlsanity.com/paper/2410.02073/) (2024)

## Built on by (most cited first, 30 of 51)

- [NeoVerse: Enhancing 4D World Model with in-the-wild Monocular Videos](https://mlsanity.com/paper/2601.00393/) (2026)
- [InfiniDepth: Arbitrary-Resolution and Fine-Grained Depth Estimation with Neural Implicit Fields](https://mlsanity.com/paper/2601.03252/) (2026)
- [4RC: 4D Reconstruction via Conditional Querying Anytime and Anywhere](https://mlsanity.com/paper/2602.10094/) (2026)
- [DriveDreamer-Policy: A Geometry-Grounded World-Action Model for Unified Generation and Planning](https://mlsanity.com/paper/2604.01765/) (2026)
- [MoGe-3: Fine-Detail Monocular Geometry Estimation with Self-Guided Sparse Volumetric Refinement](https://mlsanity.com/paper/2607.17967/) (2026)
- [PointDiT: Pixel-Space Diffusion for Monocular Geometry Estimation](https://mlsanity.com/paper/2607.02515/) (2026)
- [PromptStereo: Zero-Shot Stereo Matching via Structure and Motion Prompts](https://mlsanity.com/paper/2603.01650/) (2026)
- [Driving with DINO: Vision Foundation Features as a Unified Bridge for Sim-to-Real Generation in Autonomous Driving](https://mlsanity.com/paper/2602.06159/) (2026)
- [MoE3D: A Mixture-of-Experts Module for 3D Reconstruction](https://mlsanity.com/paper/2601.05208/) (2026)
- [Modality Forcing for Scalable Spatial Generation](https://mlsanity.com/paper/2606.13676/) (2026)
- [UniX: Unifying Autoregression and Diffusion for Chest X-Ray Understanding and Generation](https://mlsanity.com/paper/2601.11522/) (2026)
- [DAGE: Dual-Stream Architecture for Efficient and Fine-Grained Geometry Estimation](https://mlsanity.com/paper/2603.03744/) (2026)
- [FoundationGeo: Learning Spatial Pixel-Wise Fields for Monocular Metric Geometry](https://mlsanity.com/paper/2607.11588/) (2026)
- [Guardians of the Hair: Rescuing Soft Boundaries in Depth, Stereo, and Novel Views](https://mlsanity.com/paper/2601.03362/) (2026)
- [SurGe: Improved Surface Geometry in Point Maps](https://mlsanity.com/paper/2605.31577/) (2026)
- [Modeling Depth Ambiguity: A Mixture-Density Representation for Flying-Point-Free Depth Estimation](https://mlsanity.com/paper/2606.02552/) (2026)
- [How Robot Dogs See the Unseeable: Improving Visual Interpretability via Peering for Exploratory Robots](https://mlsanity.com/paper/2511.16262/) (2025)
- [Boosting Monocular Metric Depth Estimation via Bokeh Rendering](https://mlsanity.com/paper/2512.12425/) (2025)
- [Iris: Bringing Real-World Priors into Diffusion Model for Monocular Depth Estimation](https://mlsanity.com/paper/2603.16340/) (2026)
- [WildPose: A Unified Framework for Robust Pose Estimation in the Wild](https://mlsanity.com/paper/2605.12774/) (2026)
- [Unlocking Diffusion Hierarchies: Adaptive Timestep Selection for Zero-Shot Segmentation](https://mlsanity.com/paper/2606.15590/) (2026)
- [SurroundNEXO: Ego-Centric Metric Bridging for Spatially Consistent Geometry in Autonomous Driving](https://mlsanity.com/paper/2606.16960/) (2026)
- [Pixel-Space Diffusion Transformers](https://mlsanity.com/paper/2607.17585/) (2026)
- [PXDepth: Pixel-Space Modeling for Structure Preserving Monocular Depth Estimation](https://mlsanity.com/paper/2608.16984/) (2026)
- [OptiGeo: Efficient Monocular Geometry for Embodied Perception in Optically Challenging Scenes](https://mlsanity.com/paper/2608.29881/) (2026)
- [Efficient and High-Quality Depth Estimation via Pixel-Space Diffusion with Linear Attention](https://mlsanity.com/paper/2608.30129/) (2026)
- [Marigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimation](https://mlsanity.com/paper/2609.08084/) (2026)
- [Generalized Geometry Encoding Volume for Real-time Stereo Matching](https://mlsanity.com/paper/2512.06793/) (2025)
- [GVGS: Gaussian Visibility-Aware Multi-View Geometry for Accurate Surface Reconstruction](https://mlsanity.com/paper/2601.20331/) (2026)
- [Need for Speed: Zero-Shot Depth Completion with Single-Step Diffusion](https://mlsanity.com/paper/2603.10584/) (2026)

## Abstract (arXiv)

This paper presents Pixel-Perfect Depth, a monocular depth estimation model based on pixel-space diffusion generation that produces high-quality, flying-pixel-free point clouds from estimated depth maps. Current generative depth estimation models fine-tune Stable Diffusion and achieve impressive performance. However, they require a VAE to compress depth maps into latent space, which inevitably introduces textitflying pixels at edges and details. Our model addresses this challenge by directly performing diffusion generation in the pixel space, avoiding VAE-induced artifacts. To overcome the high complexity associated with pixel-space generation, we introduce two novel designs: 1) Semantics-Prompted Diffusion Transformers (SP-DiT), which incorporate semantic representations from vision foundation models into DiT to prompt the diffusion process, thereby preserving global semantic consistency while enhancing fine-grained visual details; and 2) Cascade DiT Design that progressively increases the number of tokens to further enhance efficiency and accuracy. Our model achieves the best performance among all published generative models across five benchmarks, and significantly outperforms all other models in edge-aware point cloud evaluation.