# All-in-One: Transferring Vision Foundation Models into Stereo Matching

arXiv: https://arxiv.org/abs/2412.09912 · HTML: https://mlsanity.com/paper/2412.09912/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 43 references that are in the map.
- First posted on arXiv 13 Dec 2024.

## Builds on (most cited first, 28 of 28)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [TartanAir: A Dataset to Push the Limits of Visual SLAM](https://mlsanity.com/paper/2003.14338/) (2020)
- [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018)
- [End-to-End Learning of Geometry and Context for Deep Stereo Regression](https://mlsanity.com/paper/1703.04309/) (2017)
- [RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching](https://mlsanity.com/paper/2109.07547/) (2021)
- [Paraphrasing Complex Network: Network Compression via Factor Transfer](https://mlsanity.com/paper/1802.04977/) (2018)
- [AANet: Adaptive Aggregation Network for Efficient Stereo Matching](https://mlsanity.com/paper/2004.09548/) (2020)
- [Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation](https://mlsanity.com/paper/2203.11483/) (2022)
- [Iterative Geometry Encoding Volume for Stereo Matching](https://mlsanity.com/paper/2303.06615/) (2023)
- [Hierarchical Neural Architecture Search for Deep Stereo Matching](https://mlsanity.com/paper/2010.13501/) (2020)
- [Hierarchical Deep Stereo Matching on High-resolution Images](https://mlsanity.com/paper/1912.06704/) (2019)
- [HITNet: Hierarchical Iterative Tile Refinement Network for Real-time Stereo Matching](https://mlsanity.com/paper/2007.12140/) (2020)
- [Selective-Stereo: Adaptive Frequency Information Selection for Stereo Matching](https://mlsanity.com/paper/2403.00486/) (2024)
- [Falling Things: A Synthetic Dataset for 3D Object Detection and Pose Estimation](https://mlsanity.com/paper/1804.06534/) (2018)
- [Computing the Stereo Matching Cost with a Convolutional Neural Network](https://mlsanity.com/paper/1409.4326/) (2014)
- [Knowledge Flow: Improve Upon Your Teachers](https://mlsanity.com/paper/1904.05878/) (2019)
- [Customizing Student Networks From Heterogeneous Teachers via Adaptive Knowledge Amalgamation](https://mlsanity.com/paper/1908.07121/) (2019)
- [FEED: Feature-level Ensemble for Knowledge Distillation](https://mlsanity.com/paper/1909.10754/) (2019)

## Built on by (most cited first, 16 of 16)

- [Fast-FoundationStereo: Real-Time Zero-Shot Stereo Matching](https://mlsanity.com/paper/2512.11130/) (2025)
- [FlowSeek: Optical Flow Made Easier with Depth Foundation Models and Motion Bases](https://mlsanity.com/paper/2509.05297/) (2025)
- [Stereo Any Video: Temporally Consistent Stereo Matching](https://mlsanity.com/paper/2503.05549/) (2025)
- [Lite Any Stereo: Efficient Zero-Shot Stereo Matching](https://mlsanity.com/paper/2511.16555/) (2025)
- [SMFormer: Empowering Self-supervised Stereo Matching via Foundation Models and Data Augmentation](https://mlsanity.com/paper/2604.10218/) (2026)
- [StereoVGGT: A Training-Free Visual Geometry Transformer for Stereo Vision](https://mlsanity.com/paper/2603.29368/) (2026)
- [Stextsuperscript2Mtextsuperscript2: Scalable Stereo Matching Model for Reliable Depth Estimation](https://mlsanity.com/paper/2507.13229/) (2025)
- [Towards Understanding 3D Vision: the Role of Gaussian Curvature](https://mlsanity.com/paper/2508.11825/) (2025)
- [MatchAttention: Embedding Explicit Matching Constraints into Attention for Efficient Stereo Matching](https://mlsanity.com/paper/2510.14260/) (2025)
- [Generalized Geometry Encoding Volume for Real-time Stereo Matching](https://mlsanity.com/paper/2512.06793/) (2025)
- [MLG-Stereo: ViT Based Stereo Matching with Multi-Stage Local-Global Enhancement](https://mlsanity.com/paper/2604.20393/) (2026)
- [PRISM: Synergizing Vision Foundation Models via Self-organized Expert Specialization](https://mlsanity.com/paper/2606.03444/) (2026)
- [Lite Any Stereo V2: Faster and Stronger Efficient Zero-Shot Stereo Matching](https://mlsanity.com/paper/2606.24457/) (2026)
- [Revisiting Matching Response and Swept Feature Volumes for Wide-baseline Omnidirectional Stereo](https://mlsanity.com/paper/2607.11097/) (2026)
- [M2Depth: Unifying Monocular Depth Foundation Priors with Multi-View Stereo](https://mlsanity.com/paper/2608.20788/) (2026)
- [NBS: No Bias Stereo](https://mlsanity.com/paper/2608.28933/) (2026)

## Abstract (arXiv)

As a fundamental vision task, stereo matching has made remarkable progress. While recent iterative optimization-based methods have achieved promising performance, their feature extraction capabilities still have room for improvement. Inspired by the ability of vision foundation models (VFMs) to extract general representations, in this work, we propose AIO-Stereo which can flexibly select and transfer knowledge from multiple heterogeneous VFMs to a single stereo matching model. To better reconcile features between heterogeneous VFMs and the stereo matching model and fully exploit prior knowledge from VFMs, we proposed a dual-level feature utilization mechanism that aligns heterogeneous features and transfers multi-level knowledge. Based on the mechanism, a dual-level selective knowledge transfer module is designed to selectively transfer knowledge and integrate the advantages of multiple VFMs. Experimental results show that AIO-Stereo achieves start-of-the-art performance on multiple datasets and ranks 1st on the Middlebury dataset and outperforms all the published work on the ETH3D benchmark.