# LiteMVS: Efficient Multi-View Stereo with Foundation Distillation and Expert Aggregation

arXiv: https://arxiv.org/abs/2608.03851 · HTML: https://mlsanity.com/paper/2608.03851/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Vision mixture-of-experts, one of 12 directions in Mixture-of-experts.
- Builds on 30 of its 34 references that are in the map.
- First posted on arXiv 4 Aug 2026.

## Placed in

- Mixture-of-experts / [Vision mixture-of-experts](https://mlsanity.com/moe/vision-mixture-of-experts/)

## Builds on (most cited first, 30 of 30)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning](https://mlsanity.com/paper/2306.03310/) (2023)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success](https://mlsanity.com/paper/2502.19645/) (2025)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenes](https://mlsanity.com/paper/2308.11417/) (2023)
- [RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation](https://mlsanity.com/paper/2506.18088/) (2025)
- [Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture](https://mlsanity.com/paper/1411.4734/) (2014)
- [NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video](https://mlsanity.com/paper/2104.00681/) (2021)
- [MiDaS v3.1 -- A Model Zoo for Robust Monocular Relative Depth Estimation](https://mlsanity.com/paper/2307.14460/) (2023)
- [StableNormal: Reducing Diffusion Variance for Stable and Sharp Normal](https://mlsanity.com/paper/2406.16864/) (2024)
- [Spatial Forcing: Implicit Spatial Representation Alignment for Vision-language-action Model](https://mlsanity.com/paper/2510.12276/) (2025)
- [DPSNet: End-to-end Deep Plane Sweep Stereo](https://mlsanity.com/paper/1905.00538/) (2019)
- [TransformerFusion: Monocular RGB Scene Reconstruction using Transformers](https://mlsanity.com/paper/2107.02191/) (2021)
- [SimpleRecon: 3D Reconstruction Without 3D Convolutions](https://mlsanity.com/paper/2208.14743/) (2022)
- [DeepVideoMVS: Multi-View Stereo on Video with Recurrent Spatio-Temporal Fusion](https://mlsanity.com/paper/2012.02177/) (2020)
- [MVDepthNet: Real-time Multiview Depth Estimation Neural Network](https://mlsanity.com/paper/1807.08563/) (2018)
- [MVSFormer++: Revealing the Devil in Transformer's Details for Multi-View Stereo](https://mlsanity.com/paper/2401.11673/) (2024)
- [Multi-View Stereo by Temporal Nonparametric Fusion](https://mlsanity.com/paper/1904.06397/) (2019)
- [MobileSAMv2: Faster Segment Anything to Everything](https://mlsanity.com/paper/2312.09579/) (2023)
- [VoRTX: Volumetric 3D Reconstruction With Transformers for Voxelwise View Selection and Fusion](https://mlsanity.com/paper/2112.00236/) (2021)
- [MVSAnywhere: Zero-Shot Multi-View Stereo](https://mlsanity.com/paper/2503.22430/) (2025)
- [DoubleTake: Geometry Guided Depth Estimation](https://mlsanity.com/paper/2406.18387/) (2024)
- [RPR-Net: A Point Cloud-based Rotation-aware Large Scale Place Recognition Network](https://mlsanity.com/paper/2108.12790/) (2021)

## Built on by (most cited first, 1 of 1)

- [EagleDepth: Efficient Fine-Grained Depth Estimation via Pixel Diffusion Decoder](https://mlsanity.com/paper/2610.04554/) (2026)

## Abstract (arXiv)

Real-time 3D perception is crucial for robotics, augmented reality, and embodied intelligence applications. Existing multi-view stereo (MVS) methods primarily rely on geometric correspondences, which often fail in textureless or repetitive regions, while monocular depth models leverage strong image-level priors but lack robust multi-view geometric constraints. More importantly, in robotics and embodied manipulation scenarios, high-quality 3D geometry is not only essential for static reconstruction, but also serves as a critical foundation for learning temporally consistent 4D representations. To obtain visual representations with stronger structural awareness and greater potential for spatiotemporal extension, we present LiteMVS, a lightweight multi-view depth estimation model that integrates plane-sweep geometric reasoning with strong monocular semantic and structural priors. The central idea of LiteMVS is to efficiently inject high-level monocular knowledge, obtained from lightweight segmentation models and large-scale vision foundation models, into a multi-view stereo framework. In particular, LiteMVS enriches the cost volume with semantic descriptors and employs a Mixture-of-Experts (MoE) formulation to enable adaptive geometric aggregation across depth hypotheses. Moreover, geometric priors distilled from vision foundation models further strengthen monocular guidance without increasing inference cost. Through this design, LiteMVS not only improves depth estimation and 3D reconstruction quality in static scenes, but also provides a more reliable geometric foundation for subsequent temporal modeling and 4D representation learning. Experiments on ScanNetv2 and 7-Scenes demonstrate that LiteMVS achieves high-quality depth prediction and 3D reconstruction while maintaining competitive efficiency.