# Zero-Shot Monocular Scene Flow Estimation in the Wild

arXiv: https://arxiv.org/abs/2501.10357 · HTML: https://mlsanity.com/paper/2501.10357/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 84 of its 125 references that are in the map.
- First posted on arXiv 17 Jan 2025.

## Builds on (most cited first, 30 of 84)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [Grounding Image Matching in 3D with MASt3R](https://mlsanity.com/paper/2406.09756/) (2024)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [PointCNN: Convolution On X-Transformed Points](https://mlsanity.com/paper/1801.07791/) (2018)
- [CoTracker: It is Better to Track Together](https://mlsanity.com/paper/2307.07635/) (2023)
- [MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion](https://mlsanity.com/paper/2410.03825/) (2024)
- [Kubric: A scalable dataset generator](https://mlsanity.com/paper/2203.03570/) (2022)
- [GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose](https://mlsanity.com/paper/1803.02276/) (2018)
- [PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking](https://mlsanity.com/paper/2307.15055/) (2023)
- [TAP-Vid: A Benchmark for Tracking Any Point in a Video](https://mlsanity.com/paper/2211.03726/) (2022)
- [FlowNet3D: Learning Scene Flow in 3D Point Clouds](https://mlsanity.com/paper/1806.01411/) (2018)
- [Learning to Estimate Hidden Motions with Global Motion Aggregation](https://mlsanity.com/paper/2104.02409/) (2021)
- [Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentation](https://mlsanity.com/paper/1805.09806/) (2018)
- [Particle Video Revisited: Tracking Through Occlusions Using Point Trajectories](https://mlsanity.com/paper/2204.04153/) (2022)
- [UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Loss](https://mlsanity.com/paper/1711.07837/) (2017)
- [Tracking Everything Everywhere All at Once](https://mlsanity.com/paper/2306.05422/) (2023)
- [SpatialTracker: Tracking Any 2D Pixels in 3D Space](https://mlsanity.com/paper/2404.04319/) (2024)
- [Splatt3R: Zero-shot Gaussian Splatting from Uncalibrated Image Pairs](https://mlsanity.com/paper/2408.13912/) (2024)
- [Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene Flow, Optical Flow and Stereo](https://mlsanity.com/paper/2303.01943/) (2023)
- [CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flow](https://mlsanity.com/paper/2211.10408/) (2022)
- [DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistency](https://mlsanity.com/paper/1809.01649/) (2018)
- [DynamicStereo: Consistent Dynamic Depth from Stereo Videos](https://mlsanity.com/paper/2305.02296/) (2023)

## Built on by (most cited first, 22 of 22)

- [Dream2Flow: Bridging Video Generation and Open-World Manipulation with 3D Object Flow](https://mlsanity.com/paper/2512.24766/) (2025)
- [Reconstructing 4D Spatial Intelligence: A Survey](https://mlsanity.com/paper/2507.21045/) (2025)
- [Track4World: Feedforward World-centric Dense 3D Tracking of All Pixels](https://mlsanity.com/paper/2603.02573/) (2026)
- [D2USt3R: Enhancing 3D Reconstruction for Dynamic Scenes](https://mlsanity.com/paper/2504.06264/) (2025)
- [DELTAv2: Accelerating Dense 3D Tracking](https://mlsanity.com/paper/2508.01170/) (2025)
- [Flow3r: Factored Flow Prediction for Scalable Visual Geometry Learning](https://mlsanity.com/paper/2602.20157/) (2026)
- [DeltaFlow: An Efficient Multi-frame Scene Flow Estimation Method](https://mlsanity.com/paper/2508.17054/) (2025)
- [Flow4R: Unifying 4D Reconstruction and Tracking with Scene Flow](https://mlsanity.com/paper/2602.14021/) (2026)
- [E3D-Bench: A Benchmark for End-to-End 3D Geometric Foundation Models](https://mlsanity.com/paper/2506.01933/) (2025)
- [LASER: Layer-wise Scale Alignment for Training-Free Streaming 4D Reconstruction](https://mlsanity.com/paper/2512.13680/) (2025)
- [WorldReel: 4D Video Generation with Consistent Geometry and Motion Modeling](https://mlsanity.com/paper/2512.07821/) (2025)
- [UFO-4D: Unposed Feedforward 4D Reconstruction from Two Images](https://mlsanity.com/paper/2602.24290/) (2026)
- [TrackCraft3R: Repurposing Video Diffusion Transformers for Dense 3D Tracking](https://mlsanity.com/paper/2605.12587/) (2026)
- [HuPrior3R: Incorporating Human Priors for Better 3D Dynamic Reconstruction from Monocular Videos](https://mlsanity.com/paper/2512.06368/) (2025)
- [RaLiFlow: Scene Flow Estimation with 4D Radar and LiDAR Point Clouds](https://mlsanity.com/paper/2512.10376/) (2025)
- [DePT3R: Joint Dense Point Tracking and 3D Reconstruction of Dynamic Scenes in a Single Forward Pass](https://mlsanity.com/paper/2512.13122/) (2025)
- [MotionCrafter: Dense Geometry and Motion Reconstruction with a 4D VAE](https://mlsanity.com/paper/2602.08961/) (2026)
- [UniFlow: Zero-Shot LiDAR Scene Flow for Autonomous Vehicles](https://mlsanity.com/paper/2511.18254/) (2025)
- [RAFT-MSF++: Temporal Geometry-Motion Feature Fusion for Self-Supervised Monocular Scene Flow](https://mlsanity.com/paper/2604.19349/) (2026)
- [H-Flow: Self-supervised Human Scene Flow via Physics-inspired Joint Multi-modal Learning](https://mlsanity.com/paper/2605.22629/) (2026)
- [MoSE3: Learning World-Space SE(3) at Every Pixel](https://mlsanity.com/paper/2610.03716/) (2026)
- [Interactive 4D Volumetric Liquid Forecasting under Moving-Solid Interaction](https://mlsanity.com/paper/2610.16167/) (2026)

## Abstract (arXiv)

Large models have shown generalization across datasets for many low-level vision tasks, like depth estimation, but no such general models exist for scene flow. Even though scene flow has wide potential use, it is not used in practice because current predictive models do not generalize well. We identify three key challenges and propose solutions for each. First, we create a method that jointly estimates geometry and motion for accurate prediction. Second, we alleviate scene flow data scarcity with a data recipe that affords us 1M annotated training samples across diverse synthetic scenes. Third, we evaluate different parameterizations for scene flow prediction and adopt a natural and effective parameterization. Our resulting model outperforms existing methods as well as baselines built on large-scale models in terms of 3D end-point error, and shows zero-shot generalization to the casually captured videos from DAVIS and the robotic manipulation scenes from RoboTAP. Overall, our approach makes scene flow prediction more practical in-the-wild.