SPACE-CLIPv2: Decoding Local Geometry from Frozen CLIP for Monocular Depth Estimation
New in Multimodal LLMs / vision-language models; no direction yet. Multimodal LLMs / vision-language models
Abstract (arXiv)
Vision-language foundation models such as CLIP provide strong semantic representations, but their patch tokens are not directly optimized for dense metric geometry. SPACE-CLIP showed that frozen CLIP features can support monocular depth estimation through layer-group feature fusion, yet it leaves open how neighboring CLIP tokens should be combined to recover fine local structure. We present SPACE-CLIPv2, a frozen-backbone depth decoder that aggregates fixed local neighborhoods in CLIP token space. At selected decoder stages, the model samples a fixed token stencil, predicts aggregation weights, and injects the resulting response through a gated residual update. A token-space high-pass branch further preserves shallow local contrast. On NYU Depth V2, SPACE-CLIPv2 improves over a matched SPACE-CLIP baseline, while five-seed experiments consistently favor fixed over learned-offset sampling. Zero-shot iBims-1 evaluation further improves boundary and planar-geometry measures. These results support constrained local token aggregation as a practical mechanism for decoding geometry from frozen CLIP representations.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 30 references that are in the map. First posted on arXiv 4 Oct 2026.
Builds on
22 of 30 references.
References · 22
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
- 2021Vision Transformers for Dense Predictioncited by 1,803
- 2017Deformable Convolutional Networkscited by 1,488
- 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
- 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
- 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 657
- 2020AdaBins: Depth Estimation using Adaptive Binscited by 601
- 2021DenseCLIP: Language-Guided Dense Prediction with Context-Aware Promptingcited by 532
10 more
- 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 365
- 2019DIODE: A Dense Indoor and Outdoor DEpth Datasetcited by 240
- 2022BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimationcited by 168
- 2018Evaluation of CNN-based Single-Image Depth Estimation Methodscited by 138
- 2022Can Language Understand Depth?cited by 69
- 2023Learning to Adapt CLIP for Few-Shot Monocular Depth Estimationcited by 20
- 2024PrimeDepth: Efficient Monocular Depth Estimation with a Stable Diffusion Preimagecited by 12
- 2024Structure-Centric Robust Monocular Depth Estimation via Knowledge Distillationcited by 2
- 2026SPACE-CLIP: Spatial Perception via Adaptive CLIP Embeddings for Monocular Depth Estimationcited by 2
- 2026PureCLIP-Depth: Prompt-Free and Decoder-Free Monocular Depth Estimation within CLIP Embedding Spacecited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Lightweight Prompt-Guided CLIP Adaptation for Monocular Depth Estimation
- 2025Leveraging Stable Diffusion for Monocular Depth Estimation via Image Semantic Encoding
- 2026Unlocking Dense Metric Depth Estimation in VLMs
- 2026ZipDepth: Bringing Lightweight Zero-Shot Monocular Depth Anywhere, on Any Device
- 2023Towards Zero-Shot Scale-Aware Monocular Depth Estimation
- 2025Parameter-Efficient CLIP Adaptation for 3D Understanding via Unified Tokenization
- 2026Beyond Visual Ambiguity: Guiding Robust Monocular Depth Estimation in Challenging Scenarios via Detailed Long Captions
- 2024SharpDepth: Sharpening Metric Depth Predictions Using Diffusion Distillation