mlsanityData to 7 Oct 2026

FoundationStereo: Zero-Shot Stereo Matching

arXiv 2501.09898 ↗First posted 17 Jan 2025cs.CV cs.LG cs.ROcited by 168

Not on the map. Why?

Abstract (arXiv)

Tremendous progress has been made in deep stereo matching to excel on benchmark datasets through per-domain fine-tuning. However, achieving strong zero-shot generalization - a hallmark of foundation models in other computer vision tasks - remains challenging for stereo matching. We introduce FoundationStereo, a foundation model for stereo depth estimation designed to achieve strong zero-shot generalization. To this end, we first construct a large-scale (1M stereo pairs) synthetic training dataset featuring large diversity and high photorealism, followed by an automatic self-curation pipeline to remove ambiguous samples. We then design a number of network architecture components to enhance scalability, including a side-tuning feature backbone that adapts rich monocular priors from vision foundation models to mitigate the sim-to-real gap, and long-range context reasoning for effective cost volume filtering. Together, these components lead to strong robustness and accuracy across domains, establishing a new standard in zero-shot stereo depth estimation. Project page: https://nvlabs.github.io/FoundationStereo/

Key facts. Cited by 168 papers in the map (data through 7 Oct 2026). Builds on 66 of its 91 references that are in the map. First posted on arXiv 17 Jan 2025.

Builds on

66 of 91 references.

References · 66

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2023Segment Anythingcited by 8,158
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  8. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  9. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  10. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  11. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  12. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,320
38 more
  1. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  2. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  3. 2024Depth Anything V2cited by 1,385
  4. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
  5. 2024Grounding Image Matching in 3D with MASt3Rcited by 960
  6. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 878
  7. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
  8. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 646
  9. 2022Vision Transformer Adapter for Dense Predictionscited by 564
  10. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 494
  11. 2018Pyramid Stereo Matching Networkcited by 486
  12. 2023FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objectscited by 411
  13. 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 395
  14. 2017End-to-End Learning of Geometry and Context for Deep Stereo Regressioncited by 359
  15. 2021RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matchingcited by 348
  16. 2020Virtual KITTI 2cited by 345
  17. 2023Fast Segment Anythingcited by 286
  18. 2022Unifying Flow, Stereo and Depth Estimationcited by 270
  19. 2019GA-Net: Guided Aggregation Net for End-to-end Stereo Matchingcited by 242
  20. 2018Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objectscited by 237
  21. 2019Group-wise Correlation Stereo Networkcited by 235
  22. 2023Track Anything: Segment Anything Meets Videoscited by 230
  23. 2020AANet: Adaptive Aggregation Network for Efficient Stereo Matchingcited by 202
  24. 2022Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlationcited by 197
  25. 2020Revisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformerscited by 190
  26. 2023Iterative Geometry Encoding Volume for Stereo Matchingcited by 188
  27. 2024Probing the 3D Awareness of Visual Foundation Modelscited by 175
  28. 2023Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene Flow, Optical Flow and Stereocited by 170
  29. 2020Hierarchical Neural Architecture Search for Deep Stereo Matchingcited by 166
  30. 2022CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flowcited by 165
  31. 2021CFNet: Cascade and Fused Cost Volume for Robust Stereo Matchingcited by 153
  32. 2023DynamicStereo: Consistent Dynamic Depth from Stereo Videoscited by 147
  33. 2022Attention Concatenation Volume for Accurate and Efficient Stereo Matchingcited by 130
  34. 2019Hierarchical Deep Stereo Matching on High-resolution Imagescited by 126
  35. 2022EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applicationscited by 121
  36. 2020HITNet: Hierarchical Iterative Tile Refinement Network for Real-time Stereo Matchingcited by 120
  37. 2021SMD-Nets: Stereo Mixture Density Networkscited by 118
  38. 2019Domain-invariant Stereo Matching Networkscited by 108

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 69; 2026: 99.

168 papers.

Sort:

By area:

  1. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
  2. 2026PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulationcited by 74
  3. 2025OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modelingcited by 59
  4. 2025GraspGen: A Diffusion-based Framework for 6-DOF Grasping with On-Generator Trainingcited by 32
  5. 2025BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activitiescited by 29
  6. 2025MVSAnywhere: Zero-Shot Multi-View Stereocited by 26
  7. 2026Masked Depth Modeling for Spatial Perceptioncited by 24
  8. 2025Geometry-aware 4D Video Generation for Robot Manipulationcited by 23
  9. 2025Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Surveycited by 23
  10. 2025Fast-FoundationStereo: Real-Time Zero-Shot Stereo Matchingcited by 23
  11. 2025Describe Anything Anywhere At Any Momentcited by 19
  12. 2026InfiniDepth: Arbitrary-Resolution and Fine-Grained Depth Estimation with Neural Implicit Fieldscited by 19
38 more
  1. 2026ShapeR: Robust Conditional 3D Shape Generation from Casual Capturescited by 19
  2. 2025GSFix3D: Diffusion-Guided Repair of Novel Views in Gaussian Splattingcited by 17
  3. 2025Adapt3R: Adaptive 3D Scene Representation for Domain Transfer in Imitation Learningcited by 16
  4. 2025StereoVLA: Enhancing Vision-Language-Action Models with Stereo Visioncited by 16
  5. 2025BridgeDepth: Bridging Monocular and Stereo Reasoning with Latent Alignmentcited by 15
  6. 2025GeoSVR: Taming Sparse Voxels for Geometrically Accurate Surface Reconstructioncited by 15
  7. 2025Dexterity from Smart Lenses: Multi-Fingered Robot Manipulation with In-the-Wild Human Demonstrationscited by 14
  8. 2025Object-centric 3D Motion Field for Robot Learning from Human Videoscited by 13
  9. 2025BANet: Bilateral Aggregation Network for Mobile Stereo Matchingcited by 12
  10. 2025ReSplat: Learning Recurrent Gaussian Splattingcited by 12
  11. 2025DuCos: Duality Constrained Depth Super-Resolution via Foundation Modelcited by 11
  12. 2025Manipulation as in Simulation: Enabling Accurate Geometry Perception in Robotscited by 11
  13. 2025PEEK: Guiding and Minimal Image Representations for Zero-Shot Generalization of Robot Manipulation Policiescited by 11
  14. 2025Diving into the Fusion of Monocular Priors for Generalized Stereo Matchingcited by 10
  15. 2025FlowSeek: Optical Flow Made Easier with Depth Foundation Models and Motion Basescited by 10
  16. 2026MetricAnything: Scaling Metric Depth Pretraining with Noisy Heterogeneous Sourcescited by 10
  17. 2025FindAnything: Open-Vocabulary and Object-Centric Mapping for Robot Exploration in Any Environmentcited by 8
  18. 2025StereoAdapter: Adapting Stereo Depth Estimation to Underwater Scenescited by 8
  19. 2025Maestro: Orchestrating Robotics Modules with Vision-Language Models for Zero-Shot Generalist Robotscited by 8
  20. 2025Aria Gen 2 Pilot Datasetcited by 6
  21. 2025OSMO: Open-Source Tactile Glove for Human-to-Robot Skill Transfercited by 6
  22. 2026Agentic Real2Sim: Physics-based World Modeling with Vision-Language Agentscited by 6
  23. 2026PromptStereo: Zero-Shot Stereo Matching via Structure and Motion Promptscited by 5
  24. 2026MegaFlow: Zero-Shot Large Displacement Optical Flowcited by 5
  25. 2026Reconstruction by Generation: 3D Multi-Object Scene Reconstruction from Sparse Observationscited by 5
  26. 2025GAE: Unleashing Physical Potential of VLM with Generalizable Action Expertcited by 4
  27. 2026AnyThermal: Towards Learning Universal Representations for Thermal Perceptioncited by 4
  28. 2025DEPTHOR: Depth Enhancement from a Practical Light-Weight dToF Sensor and RGB Imagecited by 3
  29. 2025Repurposing Marigold for Zero-Shot Metric Depth Estimation via Defocus Blur Cuescited by 3
  30. 2025NTIRE 2025 Challenge on HR Depth from Images of Specular and Transparent Surfacescited by 3
  31. 2025Lite Any Stereo: Efficient Zero-Shot Stereo Matchingcited by 3
  32. 2025DepthFocus: Controllable Depth Estimation for See-Through Scenescited by 3
  33. 2025Generalization Evaluation of Deep Stereo Matching Methods for UAV-Based Forestry Applicationscited by 3
  34. 2025Elastic3D: Controllable Stereo Video Conversion with Guided Latent Decodingcited by 3
  35. 2025StereoPilot: Learning Unified and Efficient Stereo Conversion via Generative Priorscited by 3
  36. 2026Large Depth Completion Model from Sparse Observationscited by 3
  37. 2026Robot-Factored World Models via Robot Renderingcited by 3
  38. 2025PanMatch: Unleashing the Potential of Large Vision Models for Unified Matching Modelscited by 2

50 most cited of 168 All 168 (JSON)