# Learning Residual Flow as Dynamic Motion from Stereo Videos

arXiv: https://arxiv.org/abs/1909.06999 · HTML: https://mlsanity.com/paper/1909.06999/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Sep 2019.

## Built on by (most cited first, 12 of 12)

- [Learning Monocular Depth in Dynamic Scenes via Instance-Aware Projection Consistency](https://mlsanity.com/paper/2102.02629/) (2021)
- [Correlate-and-Excite: Real-Time Stereo Matching via Guided Cost Volume Excitation](https://mlsanity.com/paper/2108.05773/) (2021)
- [Self-Supervised Multi-Frame Monocular Scene Flow](https://mlsanity.com/paper/2105.02216/) (2021)
- [Attentive and Contrastive Learning for Joint Depth and Motion Field Estimation](https://mlsanity.com/paper/2110.06853/) (2021)
- [Zero-Shot Monocular Scene Flow Estimation in the Wild](https://mlsanity.com/paper/2501.10357/) (2025)
- [Universal Adversarial Perturbations Through the Lens of Deep Steganography: Towards A Fourier Perspective](https://mlsanity.com/paper/2102.06479/) (2021)
- [Learning to Estimate Single-View Volumetric Flow Motions without 3D Supervision](https://mlsanity.com/paper/2302.14470/) (2023)
- [RAUM-VO: Rotational Adjusted Unsupervised Monocular Visual Odometry](https://mlsanity.com/paper/2203.07162/) (2022)
- [STDepthFormer: Predicting Spatio-temporal Depth from Video with a Self-supervised Transformer Model](https://mlsanity.com/paper/2303.01196/) (2023)
- [360 in the Wild: Dataset for Depth Prediction and View Synthesis](https://mlsanity.com/paper/2406.18898/) (2024)
- [Improved monocular depth prediction using distance transform over pre-semantic contours with self-supervised neural networks](https://mlsanity.com/paper/2605.08320/) (2026)
- [VISC: mmWave Radar Scene Flow Estimation using Pervasive Visual-Inertial Supervision](https://mlsanity.com/paper/2507.03938/) (2025)

## Abstract (arXiv)

We present a method for decomposing the 3D scene flow observed from a moving stereo rig into stationary scene elements and dynamic object motion. Our unsupervised learning framework jointly reasons about the camera motion, optical flow, and 3D motion of moving objects. Three cooperating networks predict stereo matching, camera motion, and residual flow, which represents the flow component due to object motion and not from camera motion. Based on rigid projective geometry, the estimated stereo depth is used to guide the camera motion estimation, and the depth and camera motion are used to guide the residual flow estimation. We also explicitly estimate the 3D scene flow of dynamic objects based on the residual flow and scene depth. Experiments on the KITTI dataset demonstrate the effectiveness of our approach and show that our method outperforms other state-of-the-art algorithms on the optical flow and visual odometry tasks.