mlsanityData to 14 Oct 2026

Unifying Flow, Stereo and Depth Estimation

arXiv 2211.05783 ↗First posted 10 Nov 2022cs.CVcited by 275

Not on the map. Why?

Cited by 24 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present a unified formulation and model for three motion and 3D perception tasks: optical flow, rectified stereo matching and unrectified stereo depth estimation from posed images. Unlike previous specialized architectures for each specific task, we formulate all three tasks as a unified dense correspondence matching problem, which can be solved with a single model by directly comparing feature similarities. Such a formulation calls for discriminative feature representations, which we achieve using a Transformer, in particular the cross-attention mechanism. We demonstrate that cross-attention enables integration of knowledge from another image via cross-view interactions, which greatly improves the quality of the extracted features. Our unified model naturally enables cross-task transfer since the model architecture and parameters are shared across tasks. We outperform RAFT with our unified model on the challenging Sintel dataset, and our final model that uses a few additional task-specific refinement steps outperforms or compares favorably to recent state-of-the-art methods on 10 popular flow, stereo and depth datasets, while being simpler and more efficient in terms of model design and inference speed.

Key facts. Cited by 275 papers in the map (data through 14 Oct 2026). Builds on 80 of its 118 references that are in the map. First posted on arXiv 10 Nov 2022.

Builds on

80 of 118 references.

References · 80

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2020End-to-End Object Detection with Transformerscited by 7,414
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  7. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  8. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  9. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  10. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  11. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,443
  12. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,068
38 more
  1. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,059
  2. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 968
  3. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 881
  4. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 820
  5. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 788
  6. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 784
  7. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 782
  8. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 732
  9. 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 629
  10. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 506
  11. 2018Pyramid Stereo Matching Networkcited by 489
  12. 2021ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biasescited by 484
  13. 2016Optical Flow Estimation using a Spatial Pyramid Networkcited by 424
  14. 2019Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matchingcited by 417
  15. 2016Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescuecited by 368
  16. 2017End-to-End Learning of Geometry and Context for Deep Stereo Regressioncited by 360
  17. 2020Virtual KITTI 2cited by 353
  18. 2021RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matchingcited by 353
  19. 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 342
  20. 2021GMFlow: Learning Optical Flow via Global Matchingcited by 327
  21. 2018DeepMVS: Learning Multi-view Stereopsiscited by 277
  22. 2022FlowFormer: A Transformer Architecture for Optical Flowcited by 275
  23. 2019GA-Net: Guided Aggregation Net for End-to-end Stereo Matchingcited by 244
  24. 2021Learning to Estimate Hidden Motions with Global Motion Aggregationcited by 224
  25. 2018Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentationcited by 222
  26. 2021ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Biascited by 216
  27. 2017UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Losscited by 202
  28. 2020AANet: Adaptive Aggregation Network for Efficient Stereo Matchingcited by 202
  29. 2016DeMoN: Depth and Motion Network for Learning Monocular Stereocited by 201
  30. 2022Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlationcited by 197
  31. 2020Revisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformerscited by 190
  32. 2021The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depthcited by 190
  33. 2018DeepV2D: Video to Depth with Differentiable Structure from Motioncited by 185
  34. 2018BA-Net: Dense Bundle Adjustment Networkcited by 175
  35. 2020Hierarchical Neural Architecture Search for Deep Stereo Matchingcited by 166
  36. 2019Scale-Aware Trident Networks for Object Detectioncited by 159
  37. 2021CFNet: Cascade and Fused Cost Volume for Robust Stereo Matchingcited by 154
  38. 2015Stereo Matching by Training a Convolutional Neural Network to Compare Image Patchescited by 151

50 most cited of 80 All 80 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 29; 2024: 86; 2025: 89; 2026: 70.

275 papers.

Sort:

By area:

  1. 2024MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Imagescited by 566
  2. 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 395
  3. 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 341
  4. 2024DepthSplat: Connecting Gaussian Splatting and Depthcited by 286
  5. 2023DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectorycited by 264
  6. 2024SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flowcited by 174
  7. 2025FoundationStereo: Zero-Shot Stereo Matchingcited by 173
  8. 2022CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flowcited by 166
  9. 2024Tora: Trajectory-oriented Diffusion Transformer for Video Generationcited by 164
  10. 2024GenAD: Generalized Predictive Model for Autonomous Drivingcited by 150
  11. 2024HUGS: Holistic Urban 3D Scene Understanding via Gaussian Splattingcited by 142
  12. 2024GameGen-X: Interactive Open-world Game Video Generationcited by 128
38 more
  1. 2024Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulationcited by 124
  2. 2025AdaWorld: Learning Adaptable World Models with Latent Actionscited by 114
  3. 2023Foundational Models Defining a New Era in Vision: A Survey and Outlookcited by 110
  4. 2025Cameras as Relative Positional Encodingcited by 105
  5. 2024Motion-aware 3D Gaussian Splatting for Efficient Dynamic Scene Reconstructioncited by 100
  6. 2024Selective-Stereo: Adaptive Frequency Information Selection for Stereo Matchingcited by 99
  7. 2025SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformercited by 94
  8. 2024MOFA-Video: Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion Modelcited by 89
  9. 2025JavisDiT: Joint Audio-Video Diffusion Transformer with Hierarchical Spatio-Temporal Prior Synchronizationcited by 88
  10. 2025Wan-Animate: Unified Character Animation and Replacement with Holistic Replicationcited by 80
  11. 2023GTA: A Geometry-Aware Attention Mechanism for Multi-View Transformerscited by 79
  12. 2023FlowVid: Taming Imperfect Optical Flows for Consistent Video-to-Video Synthesiscited by 79
  13. 2023Siamese Masked Autoencoderscited by 76
  14. 2024TranSplat: Generalizable 3D Gaussian Splatting from Sparse Multi-View Images with Transformerscited by 75
  15. 2023MuRF: Multi-Baseline Radiance Fieldscited by 71
  16. 2024Allegro: Open the Black Box of Commercial-Level Video Generation Modelcited by 70
  17. 2025MagicMotion: Controllable Video Generation with Dense-to-Sparse Trajectory Guidancecited by 61
  18. 2025Wan-S2V: Audio-Driven Cinematic Video Generationcited by 61
  19. 2024IGEV++: Iterative Multi-range Geometry Encoding Volumes for Stereo Matchingcited by 59
  20. 2024HiSplat: Hierarchical 3D Gaussian Splatting for Generalizable Sparse-View Reconstructioncited by 57
  21. 2024Depth Any Video with Scalable Synthetic Datacited by 57
  22. 2026SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformercited by 55
  23. 2025DEFOM-Stereo: Depth Foundation Model Based Stereo Matchingcited by 52
  24. 2025VolSplat: Rethinking Feed-Forward 3D Gaussian Splatting with Voxel-Aligned Predictioncited by 48
  25. 2023Advances in Deep Concealed Scene Understandingcited by 47
  26. 2024Stereo Anywhere: Robust Zero-Shot Deep Stereo Matching Even Where Either Stereo or Mono Failcited by 47
  27. 2023Explicit Correspondence Matching for Generalizable Neural Radiance Fieldscited by 46
  28. 2024GFlow: Recovering 4D World from Monocular Videocited by 46
  29. 2024MonoOcc: Digging into Monocular Semantic Occupancy Predictioncited by 44
  30. 2024SelfSplat: Pose-Free and 3D Prior-Free Generalizable 3D Gaussian Splattingcited by 42
  31. 2025VisualCloze: A Universal Image Generation Framework via Visual In-Context Learningcited by 39
  32. 2025UFM: A Simple Path towards Unified Dense Correspondence with Flowcited by 36
  33. 2024AnimateAnything: Consistent and Controllable Animation for Video Generationcited by 35
  34. 2024DisPose: Disentangling Pose Guidance for Controllable Human Image Animationcited by 35
  35. 2025MoVieS: Motion-Aware 4D Dynamic View Synthesis in One Secondcited by 35
  36. 2025WAFT: Warping-Alone Field Transforms for Optical Flowcited by 34
  37. 2024HDRFlow: Real-Time HDR Video Reconstruction with Large Motionscited by 31
  38. 2024DITTO: Demonstration Imitation by Trajectory Transformationcited by 30

50 most cited of 275 All 275 (JSON)