VLRC: Vision-Language Reprojection Consistency as a scalable signal for better feed-forward 3D pretraining
Not on the map. Why?
Abstract (arXiv)
Feed-forward 3D models are commonly trained using either expensive geometric supervision or self-supervised photometric objectives, both of which provide incomplete learning signals. We introduce Vision-Language Reprojection Consistency (VLRC), a scalable auxiliary objective that exploits frozen vision-language representations as semantic multi-view supervision. Given a predicted 3D reconstruction, VLRC reprojects dense vision-language features across views and enforces feature consistency between corresponding image locations, requiring no additional 3D annotations. The objective integrates seamlessly with both self-supervised monocular reconstruction and supervised-pretrained feed-forward 3D models during unlabeled adaptation. By aligning geometry with language-grounded features, VLRC not only improves depth and camera estimation but also enables more coherent multi-view semantic fusion for open-vocabulary 3D scene understanding. Experiments on indoor and outdoor benchmarks demonstrate consistent gains in 3D reconstruction accuracy and zero-shot open-vocabulary 3D semantic segmentation.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 52 of its 60 references that are in the map. First posted on arXiv 2 Jul 2026.
Builds on
52 of 60 references.
References · 52
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
- 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
- 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
- 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
38 more
- 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
- 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
- 2024Grounding Image Matching in 3D with MASt3Rcited by 960
- 2018Multi-Task Learning as Multi-Objective Optimizationcited by 809
- 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 778
- 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
- 2021Extract Free Dense Labels from CLIPcited by 580
- 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 556
- 2025Continuous 3D Perception Model with Persistent Statecited by 527
- 2021Image Segmentation Using Text and Image Promptscited by 509
- 2021PointCLIP: Point Cloud Understanding by CLIPcited by 461
- 2016YouTube-8M: A Large-Scale Video Classification Benchmarkcited by 324
- 2025MapAnything: Universal Feed-Forward Metric 3D Reconstructioncited by 295
- 2025π3: Permutation-Equivariant Visual Geometry Learningcited by 292
- 2022PLA: Language-Driven Open-Vocabulary 3D Scene Understandingcited by 176
- 2022CroCo: Self-Supervised Pre-training for 3D Vision Tasks by Cross-View Completioncited by 158
- 2021Self-Supervised Models are Continual Learnerscited by 149
- 2020Feature-metric Loss for Self-supervised Learning of Depth and Egomotioncited by 140
- 2022MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformercited by 136
- 2022PartSLIP: Low-Shot Part Segmentation for 3D Point Clouds via Pretrained Image-Language Modelscited by 117
- 2020HR-Depth: High Resolution Self-Supervised Monocular Depth Estimationcited by 115
- 2024FeatUp: A Model-Agnostic Framework for Features at Any Resolutioncited by 114
- 2025MEt3R: Measuring Multi-View Consistency in Generated Imagescited by 91
- 2020Unsupervised Monocular Depth Learning in Dynamic Scenescited by 84
- 2021Self-Supervised Monocular Depth Estimation with Internal Feature Fusioncited by 67
- 2025VL-JEPA: Joint Embedding Predictive Architecture for Vision-languagecited by 61
- 2021StructDepth: Leveraging the structural regularities for self-supervised indoor depth estimationcited by 36
- 2022RA-Depth: Resolution Adaptive Self-Supervised Monocular Depth Estimationcited by 36
- 2019Moving Indoor: Unsupervised Video Depth Learning in Challenging Environmentscited by 31
- 2025AnyCam: Learning to Recover Camera Poses and Intrinsics from Casual Videoscited by 18
- 2022MonoIndoor++:Towards Better Practice of Self-Supervised Monocular Depth Estimation for Indoor Environmentscited by 12
- 2026TIPSv2: Advancing Vision-Language Pretraining with Enhanced Patch-Text Alignmentcited by 12
- 2023MV-CLIP: Multi-View CLIP for Zero-shot 3D Shape Recognitioncited by 11
- 2023Deeper into Self-Supervised Monocular Indoor Depth Estimationcited by 7
- 2026Self-Improving 4D Perception via Self-Distillationcited by 3
- 2026SS3D: End2End Self-Supervised 3D from Web Videoscited by 1
- 2026Rebalancing gradient to improve self-supervised co-training of depth, odometry and optical flow predictionscited by 1
50 most cited of 52 All 52 (JSON)
Built on by
Not cited yet.