# Large Language Models Can Understanding Depth from Monocular Images

arXiv: https://arxiv.org/abs/2409.01133 · HTML: https://mlsanity.com/paper/2409.01133/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 10 references that are in the map.
- First posted on arXiv 2 Sep 2024.

## Builds on (most cited first, 6 of 6)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [AdaBins: Depth Estimation using Adaptive Bins](https://mlsanity.com/paper/2011.14141/) (2020)
- [Lite-Mono: A Lightweight CNN and Transformer Architecture for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2211.13202/) (2022)
- [Multi-Scale Continuous CRFs as Sequential Deep Networks for Monocular Depth Estimation](https://mlsanity.com/paper/1704.02157/) (2017)
- [Unsupervised Monocular Depth Estimation in Highly Complex Environments](https://mlsanity.com/paper/2107.13137/) (2021)
- [SwinDepth: Unsupervised Depth Estimation using Monocular Sequences via Swin Transformer and Densely Cascaded Network](https://mlsanity.com/paper/2301.06715/) (2023)

## Built on by (most cited first, 3 of 3)

- [AVA-Bench: Atomic Visual Ability Benchmark for Vision Foundation Models](https://mlsanity.com/paper/2506.09082/) (2025)
- [PureCLIP-Depth: Prompt-Free and Decoder-Free Monocular Depth Estimation within CLIP Embedding Space](https://mlsanity.com/paper/2603.16238/) (2026)
- [Skeleton-to-Image Encoding: Enabling Skeleton Representation Learning via Vision-Pretrained Models](https://mlsanity.com/paper/2603.05963/) (2026)

## Abstract (arXiv)

Monocular depth estimation is a critical function in computer vision applications. This paper shows that large language models (LLMs) can effectively interpret depth with minimal supervision, using efficient resource utilization and a consistent neural network architecture. We introduce LLM-MDE, a multimodal framework that deciphers depth through language comprehension. Specifically, LLM-MDE employs two main strategies to enhance the pretrained LLM's capability for depth estimation: cross-modal reprogramming and an adaptive prompt estimation module. These strategies align vision representations with text prototypes and automatically generate prompts based on monocular images, respectively. Comprehensive experiments on real-world MDE datasets confirm the effectiveness and superiority of LLM-MDE, which excels in few-/zero-shot tasks while minimizing resource use. The source code is available.