# DepthLab: From Partial to Complete

arXiv: https://arxiv.org/abs/2412.18153 · HTML: https://mlsanity.com/paper/2412.18153/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 73 of its 88 references that are in the map.
- First posted on arXiv 24 Dec 2024.

## Builds on (most cited first, 30 of 73)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks](https://mlsanity.com/paper/2401.14159/) (2024)
- [Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation](https://mlsanity.com/paper/2311.17117/) (2023)
- [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020)
- [AdaBins: Depth Estimation using Adaptive Bins](https://mlsanity.com/paper/2011.14141/) (2020)
- [Semantic Scene Completion from a Single Depth Image](https://mlsanity.com/paper/1611.08974/) (2016)
- [Depth Pro: Sharp Monocular Metric Depth in Less Than a Second](https://mlsanity.com/paper/2410.02073/) (2024)
- [ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis](https://mlsanity.com/paper/2409.02048/) (2024)
- [Gaussian Grouping: Segment and Edit Anything in 3D Scenes](https://mlsanity.com/paper/2312.00732/) (2023)
- [Decomposing NeRF for Editing via Feature Field Distillation](https://mlsanity.com/paper/2205.15585/) (2022)
- [Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Image](https://mlsanity.com/paper/2307.10984/) (2023)
- [Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scans](https://mlsanity.com/paper/2110.04994/) (2021)
- [GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting](https://mlsanity.com/paper/2311.14521/) (2023)
- [MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model](https://mlsanity.com/paper/2311.16498/) (2023)
- [Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation](https://mlsanity.com/paper/2312.02145/) (2023)
- [GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image](https://mlsanity.com/paper/2403.12013/) (2024)
- [Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Models](https://mlsanity.com/paper/2303.11989/) (2023)
- [DIODE: A Dense Indoor and Outdoor DEpth Dataset](https://mlsanity.com/paper/1908.00463/) (2019)
- [LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenes](https://mlsanity.com/paper/2311.13384/) (2023)
- [Learning to Recover 3D Scene Shape from a Single Image](https://mlsanity.com/paper/2012.09365/) (2020)

## Built on by (most cited first, 27 of 27)

- [DiST-4D: Disentangled Spatiotemporal Diffusion with Metric Depth for 4D Driving Scene Generation](https://mlsanity.com/paper/2503.15208/) (2025)
- [Depth Anything with Any Prior](https://mlsanity.com/paper/2505.10565/) (2025)
- [OmniNWM: Omniscient Driving Navigation World Models](https://mlsanity.com/paper/2510.18313/) (2025)
- [WonderTurbo: Generating Interactive 3D World in 0.72 Seconds](https://mlsanity.com/paper/2504.02261/) (2025)
- [InfiniDepth: Arbitrary-Resolution and Fine-Grained Depth Estimation with Neural Implicit Fields](https://mlsanity.com/paper/2601.03252/) (2026)
- [WorldWeaver: Generating Long-Horizon Video Worlds via Rich Perception](https://mlsanity.com/paper/2508.15720/) (2025)
- [VPGS-SLAM: Voxel-based Progressive 3D Gaussian SLAM in Large-Scale Scenes](https://mlsanity.com/paper/2505.18992/) (2025)
- [DreamCube: 3D Panorama Generation via Multi-plane Synchronization](https://mlsanity.com/paper/2506.17206/) (2025)
- [Depth Anything at Any Condition](https://mlsanity.com/paper/2507.01634/) (2025)
- [DuCos: Duality Constrained Depth Super-Resolution via Foundation Model](https://mlsanity.com/paper/2503.04171/) (2025)
- [MetricAnything: Scaling Metric Depth Pretraining with Noisy Heterogeneous Sources](https://mlsanity.com/paper/2601.22054/) (2026)
- [MuDG: Taming Multi-modal Diffusion with Gaussian Splatting for Urban Scene Reconstruction](https://mlsanity.com/paper/2503.10604/) (2025)
- [4D Driving Scene Generation With Stereo Forcing](https://mlsanity.com/paper/2509.20251/) (2025)
- [Video Understanding: From Geometry and Semantics to Unified Models](https://mlsanity.com/paper/2603.17840/) (2026)
- [LuciBot: Automated Robot Policy Learning from Generated Videos](https://mlsanity.com/paper/2503.09871/) (2025)
- [Large Depth Completion Model from Sparse Observations](https://mlsanity.com/paper/2605.30115/) (2026)
- [DEPTHOR: Depth Enhancement from a Practical Light-Weight dToF Sensor and RGB Image](https://mlsanity.com/paper/2504.01596/) (2025)
- [VecGlypher: Unified Vector Glyph Generation with Language Models](https://mlsanity.com/paper/2602.21461/) (2026)
- [StreamTTO: Efficient Online Test-Time Optimization for Video Depth Completion](https://mlsanity.com/paper/2603.01765/) (2026)
- [LiDAR Prompted Spatio-Temporal Multi-View Stereo for Autonomous Driving](https://mlsanity.com/paper/2603.03765/) (2026)
- [WorldAct: Activating Monolithic 3D Worlds into Interactive-Ready Object-Centric Scenes](https://mlsanity.com/paper/2605.15843/) (2026)
- [Perfecting Depth: Uncertainty-Aware Enhancement of Metric Depth](https://mlsanity.com/paper/2506.04612/) (2025)
- [SurroundNEXO: Ego-Centric Metric Bridging for Spatially Consistent Geometry in Autonomous Driving](https://mlsanity.com/paper/2606.16960/) (2026)
- [LVD-GS: Gaussian Splatting SLAM for Dynamic Scenes via Hierarchical Explicit-Implicit Representation Collaboration Rendering](https://mlsanity.com/paper/2510.22669/) (2025)
- [PTC-Depth: Pose-Refined Monocular Depth Estimation with Temporal Consistency](https://mlsanity.com/paper/2604.01791/) (2026)
- [Dense Metric Depth Completion from Sparse Direct Time-of-Flight Sensors](https://mlsanity.com/paper/2608.04737/) (2026)
- [Pumpire: Unified Benchmark for Metric Distance Estimation](https://mlsanity.com/paper/2610.12423/) (2026)

## Abstract (arXiv)

Missing values remain a common challenge for depth data across its wide range of applications, stemming from various causes like incomplete data acquisition and perspective alteration. This work bridges this gap with DepthLab, a foundation depth inpainting model powered by image diffusion priors. Our model features two notable strengths: (1) it demonstrates resilience to depth-deficient regions, providing reliable completion for both continuous areas and isolated points, and (2) it faithfully preserves scale consistency with the conditioned known depth when filling in missing values. Drawing on these advantages, our approach proves its worth in various downstream tasks, including 3D scene inpainting, text-to-3D scene generation, sparse-view reconstruction with DUST3R, and LiDAR depth completion, exceeding current solutions in both numerical performance and visual quality. Our project page with source code is available at https://johanan528.github.io/depthlab_web/.