mlsanityData to 7 Oct 2026

Prompting Depth Anything for 4K Resolution Accurate Metric Depth Estimation

arXiv 2412.14015 ↗First posted 18 Dec 2024cs.CVcited by 73

Not on the map. Why?

Abstract (arXiv)

Prompts play a critical role in unleashing the power of language and vision foundation models for specific tasks. For the first time, we introduce prompting into depth foundation models, creating a new paradigm for metric depth estimation termed Prompt Depth Anything. Specifically, we use a low-cost LiDAR as the prompt to guide the Depth Anything model for accurate metric depth output, achieving up to 4K resolution. Our approach centers on a concise prompt fusion design that integrates the LiDAR at multiple scales within the depth decoder. To address training challenges posed by limited datasets containin both LiDAR depth and precise GT depth, we propose a scalable data pipeline that includes synthetic data LiDAR simulation and real data pseudo GT depth generation. Our approach sets new state-of-the-arts on the ARKitScenes and ScanNet++ datasets and benefits downstream applications, including 3D reconstruction and generalized robotic grasping.

Key facts. Cited by 73 papers in the map (data through 7 Oct 2026). Builds on 67 of its 86 references that are in the map. First posted on arXiv 18 Dec 2024.

Builds on

67 of 86 references.

References · 67

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2023Segment Anythingcited by 8,158
  7. 2023Visual Instruction Tuningcited by 7,792
  8. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  9. 2022Scalable Diffusion Models with Transformerscited by 5,931
  10. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  11. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  12. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
38 more
  1. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  2. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  3. 2021Vision Transformers for Dense Predictioncited by 1,789
  4. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  5. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  6. 2024Depth Anything V2cited by 1,385
  7. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
  8. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  9. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 747
  10. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 731
  11. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 721
  12. 2023Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fieldscited by 667
  13. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 646
  14. 2020AdaBins: Depth Estimation using Adaptive Binscited by 596
  15. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
  16. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 494
  17. 2019BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networkscited by 487
  18. 2014Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecturecited by 483
  19. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 442
  20. 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 395
  21. 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 389
  22. 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 375
  23. 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 357
  24. 2024GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Imagecited by 283
  25. 2021Function4D: Real-time Human Volumetric Capture from Very Sparse Consumer RGBD Sensorscited by 246
  26. 2020Learning to Recover 3D Scene Shape from a Single Imagecited by 214
  27. 2023Iterative Geometry Encoding Volume for Stereo Matchingcited by 188
  28. 2023MiDaS v3.1 -- A Model Zoo for Robust Monocular Relative Depth Estimationcited by 173
  29. 2022BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimationcited by 166
  30. 2024Lotus: Diffusion-based Visual Foundation Model for High-quality Dense Predictioncited by 163
  31. 2022SHIFT: A Synthetic Driving Dataset for Continuous Multi-Task Domain Adaptationcited by 146
  32. 2018Learning Depth with Convolutional Spatial Propagation Networkcited by 135
  33. 2022Attention Concatenation Volume for Accurate and Efficient Stereo Matchingcited by 130
  34. 2019Learning Guided Convolutional Network for Depth Completioncited by 111
  35. 2019CSPN++: Learning Context and Resource Aware Convolutional Spatial Propagation Networks for Depth Completioncited by 110
  36. 2023Towards Zero-Shot Scale-Aware Monocular Depth Estimationcited by 100
  37. 2022LocalBins: Improving Depth Estimation by Learning Local Distributionscited by 92
  38. 2023CompletionFormer: Depth Completion with Convolutions and Vision Transformerscited by 92

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 33; 2026: 39.

73 papers.

Sort:

By area:

  1. 2025MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Detailscited by 250
  2. 2025Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformerscited by 47
  3. 2024EnvGS: Modeling View-Dependent Appearance with Environment Gaussiancited by 34
  4. 2025Depth Anything with Any Priorcited by 32
  5. 2026Masked Depth Modeling for Spatial Perceptioncited by 24
  6. 2025MonSter++: Unified Stereo Matching, Multi-view Stereo, and Real-time Stereo with Monodepth Priorscited by 19
  7. 2026InfiniDepth: Arbitrary-Resolution and Fine-Grained Depth Estimation with Neural Implicit Fieldscited by 19
  8. 2025iTACO: Interactable Digital Twins of Articulated Objects from Casually Captured RGBD Videoscited by 18
  9. 2025Multi-view Reconstruction via SfM-guided Monocular Depth Estimationcited by 17
  10. 2025FlashDepth: Real-time Streaming Video Depth Estimation at 2K Resolutioncited by 14
  11. 2025BANet: Bilateral Aggregation Network for Mobile Stereo Matchingcited by 12
  12. 2025DreamCube: 3D Panorama Generation via Multi-plane Synchronizationcited by 12
38 more
  1. 2025DuCos: Duality Constrained Depth Super-Resolution via Foundation Modelcited by 11
  2. 2025Depth Anything at Any Conditioncited by 11
  3. 2025Manipulation as in Simulation: Enabling Accurate Geometry Perception in Robotscited by 11
  4. 2026MetricAnything: Scaling Metric Depth Pretraining with Noisy Heterogeneous Sourcescited by 10
  5. 2025UniGeo: Taming Video Diffusion for Unified Consistent Geometry Estimationcited by 9
  6. 2025Diffusion Knows Transparency: Repurposing Video Diffusion for Transparent Object Depth and Normal Estimationcited by 9
  7. 2026EgoGrasp: World-Space Hand-Object Interaction Estimation from Egocentric Videoscited by 7
  8. 2025Image as an IMU: Estimating Camera Motion from a Single Motion-Blurred Imagecited by 5
  9. 2026PromptStereo: Zero-Shot Stereo Matching via Structure and Motion Promptscited by 5
  10. 2026Any to Full: Prompting Depth Anything for Depth Completion in One Stagecited by 5
  11. 2025GAE: Unleashing Physical Potential of VLM with Generalizable Action Expertcited by 4
  12. 2025BoxFusion: Reconstruction-Free Open-Vocabulary 3D Object Detection via Real-Time Multi-View Box Fusioncited by 3
  13. 2025Region-aware Depth Scale Adaptation with Sparse Measurementscited by 3
  14. 2025PALMS+: Modular Image-Based Floor Plan Localization Leveraging Depth Foundation Modelcited by 3
  15. 2026Depth Completion as Parameter-Efficient Test-Time Adaptationcited by 3
  16. 2026StreamTTO: Efficient Online Test-Time Optimization for Video Depth Completioncited by 3
  17. 2026Large Depth Completion Model from Sparse Observationscited by 3
  18. 2025Towards Depth Foundation Model: Recent Trends in Vision-Based Depth Estimationcited by 2
  19. 2025Propagating Sparse Depth via Depth Foundation Model for Out-of-Distribution Depth Completioncited by 2
  20. 2025Robust Single-shot Structured Light 3D Imaging via Neural Feature Decodingcited by 2
  21. 2026Pixel-Perfect Visual Geometry Estimationcited by 2
  22. 2026LiDAR Prompted Spatio-Temporal Multi-View Stereo for Autonomous Drivingcited by 2
  23. 2026GGPT: Geometry Grounded Point Transformercited by 2
  24. 2025PFDepth: Heterogeneous Pinhole-Fisheye Joint Depth Estimation via Distortion-aware Gaussian-Splatted Volumetric Fusioncited by 1
  25. 2025DEPTHOR++: Robust Depth Enhancement from a Real-World Lightweight dToF and RGB Guidancecited by 1
  26. 2025CogStereo: Neural Stereo Matching with Implicit Spatial Cognition Embeddingcited by 1
  27. 2025Splat-SAP: Feed-Forward Gaussian Splatting for Human-Centered Scene with Scale-Aware Point Map Reconstructioncited by 1
  28. 2025StarryGazer: Leveraging Monocular Depth Estimation Models for Domain-Agnostic Single Depth Image Completioncited by 1
  29. 2026Language as Prior, Vision as Calibration: Metric Scale Recovery for Monocular Depth Estimationcited by 1
  30. 2026EmbodMocap: In-the-Wild 4D Human-Scene Reconstruction for Embodied Agentscited by 1
  31. 2026Local Precise Refinement: A Dual-Gated Mixture-of-Experts for Enhancing Foundation Model Generalization against Spectral Shiftscited by 1
  32. 2026The Midas Touch for Metric Depthcited by 1
  33. 2026Sparse-LiDAR Prompting of Monocular Geometry Foundations: An Empirical Study Toward Long-Range Driving Depthcited by 1
  34. 2026SurroundNEXO: Ego-Centric Metric Bridging for Spatially Consistent Geometry in Autonomous Drivingcited by 1
  35. 2026FlowObject: Flow Steering for Bridging Generative Priors and Reconstruction Fidelitycited by 1
  36. 2026The Multipath Blind Spot: K-Agnostic Robust Calibration for Sparse-Anchor Metric Depth from Frozen Foundationscited by 1
  37. 2026OmniPoint: Universal Monocular Metric Pointcloud from Any Cameracited by 1
  38. 2025Survey on Monocular Metric Depth Estimationcited by 0

50 most cited of 73 All 73 (JSON)