# Optical Flow in Mostly Rigid Scenes

arXiv: https://arxiv.org/abs/1705.01352 · HTML: https://mlsanity.com/paper/1705.01352/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 May 2017.

## Built on by (most cited first, 20 of 20)

- [Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene Flow, Optical Flow and Stereo](https://mlsanity.com/paper/2303.01943/) (2023)
- [Learning to Segment Rigid Motions from Two Frames](https://mlsanity.com/paper/2101.03694/) (2021)
- [VOLDOR: Visual Odometry from Log-logistic Dense Optical flow Residuals](https://mlsanity.com/paper/2104.06789/) (2021)
- [M-FUSE: Multi-frame Fusion for Scene Flow Estimation](https://mlsanity.com/paper/2207.05704/) (2022)
- [Dimensions of Motion: Monocular Prediction through Flow Subspaces](https://mlsanity.com/paper/2112.01502/) (2021)
- [SemARFlow: Injecting Semantics into Unsupervised Optical Flow Estimation for Autonomous Driving](https://mlsanity.com/paper/2303.06209/) (2023)
- [UnSAMFlow: Unsupervised Optical Flow Guided by Segment Anything Model](https://mlsanity.com/paper/2405.02608/) (2024)
- [SplatFlow: Learning Multi-frame Optical Flow via Splatting](https://mlsanity.com/paper/2306.08887/) (2023)
- [On Moving Object Segmentation from Monocular Video with Transformers](https://mlsanity.com/paper/2411.19141/) (2024)
- [NudgeSeg: Zero-Shot Object Segmentation by Repeated Physical Interaction](https://mlsanity.com/paper/2109.13859/) (2021)
- [Joint Prediction of Monocular Depth and Structure using Planar and Parallax Geometry](https://mlsanity.com/paper/2207.06351/) (2022)
- [MonoPP: Metric-Scaled Self-Supervised Monocular Depth Estimation by Planar-Parallax Geometry in Automotive Applications](https://mlsanity.com/paper/2411.19717/) (2024)
- [Self-Supervised Multi-Frame Neural Scene Flow](https://mlsanity.com/paper/2403.16116/) (2024)
- [MotionScape: A Motion-Stratified UAV Video Benchmark for World Modeling and Future Video Generation](https://mlsanity.com/paper/2604.07991/) (2026)
- [Unsupervised Learning of Depth, Camera Pose and Optical Flow from Monocular Video](https://mlsanity.com/paper/2205.09821/) (2022)
- [FlowNAS: Neural Architecture Search for Optical Flow Estimation](https://mlsanity.com/paper/2207.01271/) (2022)
- [Efficient 3D Reconstruction, Streaming and Visualization of Static and Dynamic Scene Parts for Multi-client Live-telepresence in Large-scale Environments](https://mlsanity.com/paper/2211.14310/) (2022)
- [DepthP+P: Metric Accurate Monocular Depth Estimation using Planar and Parallax](https://mlsanity.com/paper/2301.02092/) (2023)
- [Unsupervised Region-Growing Network for Object Segmentation in Atmospheric Turbulence](https://mlsanity.com/paper/2311.03572/) (2023)
- [MaskFlow: Object-Aware Motion Estimation](https://mlsanity.com/paper/2311.12476/) (2023)

## Abstract (arXiv)

The optical flow of natural scenes is a combination of the motion of the observer and the independent motion of objects. Existing algorithms typically focus on either recovering motion and structure under the assumption of a purely static world or optical flow for general unconstrained scenes. We combine these approaches in an optical flow algorithm that estimates an explicit segmentation of moving objects from appearance and physical constraints. In static regions we take advantage of strong constraints to jointly estimate the camera motion and the 3D structure of the scene over multiple frames. This allows us to also regularize the structure instead of the motion. Our formulation uses a Plane+Parallax framework, which works even under small baselines, and reduces the motion estimation to a one-dimensional search problem, resulting in more accurate estimation. In moving regions the flow is treated as unconstrained, and computed with an existing optical flow method. The resulting Mostly-Rigid Flow (MR-Flow) method achieves state-of-the-art results on both the MPI-Sintel and KITTI-2015 benchmarks.