# SteeredMarigold: Steering Diffusion Towards Depth Completion of Largely Incomplete Depth Maps

arXiv: https://arxiv.org/abs/2409.10202 · HTML: https://mlsanity.com/paper/2409.10202/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 50 references that are in the map.
- First posted on arXiv 16 Sep 2024.

## Builds on (most cited first, 30 of 35)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [Imagic: Text-Based Real Image Editing with Diffusion Models](https://mlsanity.com/paper/2210.09276/) (2022)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)
- [Blended Diffusion for Text-driven Editing of Natural Images](https://mlsanity.com/paper/2111.14818/) (2021)
- [ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth](https://mlsanity.com/paper/2302.12288/) (2023)
- [ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2108.02938/) (2021)
- [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020)
- [UniDepth: Universal Monocular Metric Depth Estimation](https://mlsanity.com/paper/2403.18913/) (2024)
- [Virtual Worlds as Proxy for Multi-Object Tracking Analysis](https://mlsanity.com/paper/1605.06457/) (2016)
- [Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation](https://mlsanity.com/paper/2312.02145/) (2023)
- [Sparsity Invariant CNNs](https://mlsanity.com/paper/1708.06500/) (2017)
- [Diffusion models as plug-and-play priors](https://mlsanity.com/paper/2206.09012/) (2022)
- [Non-Local Spatial Propagation Network for Depth Completion](https://mlsanity.com/paper/2007.10042/) (2020)
- [DDP: Diffusion Model for Dense Visual Prediction](https://mlsanity.com/paper/2303.17559/) (2023)
- [PENet: Towards Precise and Efficient Image Guided Depth Completion](https://mlsanity.com/paper/2103.00783/) (2021)
- [Learning Guided Convolutional Network for Depth Completion](https://mlsanity.com/paper/1908.01238/) (2019)
- [Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Network](https://mlsanity.com/paper/1808.00150/) (2018)
- [CSPN++: Learning Context and Resource Aware Convolutional Spatial Propagation Networks for Depth Completion](https://mlsanity.com/paper/1911.05377/) (2019)
- [In Defense of Classical Image Processing: Fast Depth Completion on the CPU](https://mlsanity.com/paper/1802.00036/) (2018)
- [CompletionFormer: Depth Completion with Convolutions and Vision Transformers](https://mlsanity.com/paper/2304.13030/) (2023)
- [DiffusionDepth: Diffusion Denoising Approach for Monocular Depth Estimation](https://mlsanity.com/paper/2303.05021/) (2023)
- [Bilateral Propagation Network for Depth Completion](https://mlsanity.com/paper/2403.11270/) (2024)
- [LRRU: Long-short Range Recurrent Updating Networks for Depth Completion](https://mlsanity.com/paper/2310.08956/) (2023)

## Built on by (most cited first, 7 of 7)

- [Marigold: Affordable Adaptation of Diffusion-Based Image Generators for Image Analysis](https://mlsanity.com/paper/2505.09358/) (2025)
- [Marigold-DC: Zero-Shot Monocular Depth Completion with Guided Diffusion](https://mlsanity.com/paper/2412.13389/) (2024)
- [Event-Driven Dynamic Scene Depth Completion](https://mlsanity.com/paper/2505.13279/) (2025)
- [DEPTHOR: Depth Enhancement from a Practical Light-Weight dToF Sensor and RGB Image](https://mlsanity.com/paper/2504.01596/) (2025)
- [Depth Completion as Parameter-Efficient Test-Time Adaptation](https://mlsanity.com/paper/2602.14751/) (2026)
- [DEPTHOR++: Robust Depth Enhancement from a Real-World Lightweight dToF and RGB Guidance](https://mlsanity.com/paper/2509.26498/) (2025)
- [Need for Speed: Zero-Shot Depth Completion with Single-Step Diffusion](https://mlsanity.com/paper/2603.10584/) (2026)

## Abstract (arXiv)

Even if the depth maps captured by RGB-D sensors deployed in real environments are often characterized by large areas missing valid depth measurements, the vast majority of depth completion methods still assumes depth values covering all areas of the scene. To address this limitation, we introduce SteeredMarigold, a training-free, zero-shot depth completion method capable of producing metric dense depth, even for largely incomplete depth maps. SteeredMarigold achieves this by using the available sparse depth points as conditions to steer a denoising diffusion probabilistic model. Our method outperforms relevant top-performing methods on the NYUv2 dataset, in tests where no depth was provided for a large area, achieving state-of-art performance and exhibiting remarkable robustness against depth map incompleteness. Our source code is publicly available at https://steeredmarigold.github.io.