mlsanityData to 14 Oct 2026

Can Language Understand Depth?

arXiv 2207.01077 ↗First posted 3 Jul 2022cs.CV cs.CL cs.MMcited by 69

Not on the map. Why?

Abstract (arXiv)

Besides image classification, Contrastive Language-Image Pre-training (CLIP) has accomplished extraordinary success for a wide range of vision tasks, including object-level and 3D space understanding. However, it's still challenging to transfer semantic knowledge learned from CLIP into more intricate tasks of quantified targets, such as depth estimation with geometric information. In this paper, we propose to apply CLIP for zero-shot monocular depth estimation, named DepthCLIP. We found that the patches of the input image could respond to a certain semantic distance token and then be projected to a quantified depth bin for coarse estimation. Without any training, our DepthCLIP surpasses existing unsupervised methods and even approaches the early fully-supervised networks. To our best knowledge, we are the first to conduct zero-shot adaptation from the semantic language knowledge to quantified downstream tasks and perform zero-shot monocular depth estimation. We hope our work could cast a light on future research. The code is available at https://github.com/Adonis-galaxy/DepthCLIP.

Key facts. Cited by 69 papers in the map (data through 14 Oct 2026). Builds on 26 of its 35 references that are in the map. First posted on arXiv 3 Jul 2022.

Builds on

26 of 35 references.

References · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2020End-to-End Object Detection with Transformerscited by 7,414
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  7. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  8. 2021Learning to Prompt for Vision-Language Modelscited by 2,281
  9. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  10. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,026
  11. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 820
  12. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 723
14 more
  1. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 614
  2. 2021DenseCLIP: Language-Guided Dense Prediction with Context-Aware Promptingcited by 532
  3. 2021PointCLIP: Point Cloud Understanding by CLIPcited by 463
  4. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 447
  5. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modelingcited by 431
  6. 2019From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimationcited by 382
  7. 2020SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimationcited by 204
  8. 2018Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraintscited by 179
  9. 2022DepthFormer: Exploiting Long-Range Correlation and Local Information for Accurate Monocular Depth Estimationcited by 118
  10. 2021VT-CLIP: Enhancing Vision-Language Models with Visual-guided Textscited by 42
  11. 2021Dual-stream Network for Visual Recognitioncited by 33
  12. 2021PLNet: Plane and Line Priors for Unsupervised Indoor Depth Estimationcited by 12
  13. 2018Inferring Point Clouds from Single Monocular Images by Depth Intermediationcited by 7
  14. 2022End-to-end Learning for Joint Depth and Image Reconstruction from Diffracted Rotationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 6; 2023: 15; 2024: 29; 2025: 9; 2026: 10.

69 papers.

Sort:

By area:

  1. 2023Prompt, Generate, then Cache: Cascade of Foundation Models makes Strong Few-shot Learnerscited by 152
  2. 2024SpatialBot: Precise Spatial Understanding with Vision Language Modelscited by 152
  3. 2022CALIP: Zero-Shot Enhancement of CLIP with Parameter-free Attentioncited by 103
  4. 2022EDA: Explicit Text-Decoupling and Dense Alignment for 3D Visual Groundingcited by 99
  5. 2024Binding Touch to Everything: Learning Unified Multimodal Tactile Representationscited by 95
  6. 2023Not All Features Matter: Enhancing Few-shot CLIP with Adaptive Prior Refinementcited by 84
  7. 2023ViewRefer: Grasp the Multi-view Knowledge for 3D Visual Grounding with GPT and Prototype Guidancecited by 70
  8. 2024Two Effects, One Trigger: On the Modality Gap, Object Bias, and Information Imbalance in Contrastive Vision-Language Modelscited by 57
  9. 2023Deception Detection from Linguistic and Physiological Data Streams Using Bimodal Convolutional Neural Networkscited by 56
  10. 2023PiMAE: Point Cloud and Image Interactive Masked Autoencoders for 3D Object Detectioncited by 53
  11. 2022Dual Modality Prompt Tuning for Vision-Language Pre-Trained Modelcited by 52
  12. 2023Joint-MAE: 2D-3D Joint Masked Autoencoders for 3D Point Cloud Pre-trainingcited by 52
38 more
  1. 2024Dual Prototype Evolving for Test-Time Generalization of Vision-Language Modelscited by 38
  2. 2022TiG-BEV: Multi-view BEV 3D Object Detection via Target Inner-Geometry Learningcited by 37
  3. 2023Parameter is Not All You Need: Starting from Non-Parametric Networks for 3D Point Cloud Analysiscited by 34
  4. 2023EventBind: Learning a Unified Representation to Bind Them All for Event-based Open-world Understandingcited by 34
  5. 2024WorDepth: Variational Language Prior for Monocular Depth Estimationcited by 34
  6. 2022LidarCLIP or: How I Learned to Talk to Point Cloudscited by 33
  7. 2024Transfer CLIP for Generalizable Image Denoisingcited by 29
  8. 2024RSA: Resolving Scale Ambiguities in Monocular Depth Estimators through Language Descriptionscited by 21
  9. 2025Jasmine: Harnessing Diffusion Prior for Self-supervised Depth Estimationcited by 21
  10. 2023Learning to Adapt CLIP for Few-Shot Monocular Depth Estimationcited by 20
  11. 2024Enhance Image-to-Image Generation with LLaVA-generated Promptscited by 20
  12. 2024Teach CLIP to Develop a Number Sense for Ordinal Regressioncited by 20
  13. 2024Harnessing LLMs for API Interactions: A Framework for Classification and Synthetic Data Generationcited by 17
  14. 2024CLIPose: Category-Level Object Pose Estimation with Pre-trained Vision-Language Knowledgecited by 16
  15. 2024Evaluating Modern Approaches in 3D Scene Reconstruction: NeRF vs Gaussian-Based Methodscited by 16
  16. 2025Vision-Language Embodiment for Monocular Depth Estimationcited by 16
  17. 2024ScaleDepth: Decomposing Metric Depth Estimation into Scale Prediction and Relative Depth Estimationcited by 15
  18. 2024NEVLP: Noise-Robust Framework for Efficient Vision-Language Pre-trainingcited by 15
  19. 2023Benchmarking Zero-Shot Recognition with Vision-Language Models: Challenges on Granularity and Specificitycited by 14
  20. 2024NeuroBind: Towards Unified Multimodal Representations for Neural Signalscited by 13
  21. 2023LDP: Language-driven Dual-Pixel Image Defocus Deblurring Networkcited by 11
  22. 2024POV: Prompt-Oriented View-Agnostic Learning for Egocentric Hand-Object Interaction in the Multi-View Worldcited by 10
  23. 2024Text-Driven Traffic Anomaly Detection with Temporal High-Frequency Modeling in Driving Videoscited by 9
  24. 2024Style Transfer: From Stitching to Neural Networkscited by 8
  25. 2024CLIP Can Understand Depthcited by 7
  26. 2024Towards Comprehensive Multimodal Perception: Introducing the Touch-Language-Vision Datasetcited by 6
  27. 2025Hybrid-grained Feature Aggregation with Coarse-to-fine Language Guidance for Self-supervised Monocular Depth Estimationcited by 6
  28. 2025Test-Time Distillation for Continual Model Adaptationcited by 5
  29. 2023LAMP: Leveraging Language Prompts for Multi-person Pose Estimationcited by 4
  30. 2023Consistency Prototype Module and Motion Compensation for Few-Shot Action Recognition (CLIP-CPM2C)cited by 4
  31. 2024TiGDistill-BEV: Multi-view BEV 3D Object Detection via Target Inner-Geometry Learning Distillationcited by 4
  32. 2024CoLA: Conditional Dropout and Language-driven Robust Dual-modal Salient Object Detectioncited by 3
  33. 2025Differential Contrastive Training for Gaze Estimationcited by 3
  34. 2023The Potential of Vision-Language Models for Content Moderation of Children's Videoscited by 2
  35. 2024SM4Depth: Seamless Monocular Metric Depth Estimation across Multiple Cameras and Scenes by One Modelcited by 2
  36. 2024Enhancing Exchange Rate Forecasting with Explainable Deep Learning Modelscited by 2
  37. 2024Ranking-aware adapter for text-driven image ordering with CLIPcited by 2
  38. 2026SPACE-CLIP: Spatial Perception via Adaptive CLIP Embeddings for Monocular Depth Estimationcited by 2

50 most cited of 69 All 69 (JSON)