# EMR-MSF: Self-Supervised Recurrent Monocular Scene Flow Exploiting Ego-Motion Rigidity

arXiv: https://arxiv.org/abs/2309.01296 · HTML: https://mlsanity.com/paper/2309.01296/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 63 references that are in the map.
- First posted on arXiv 4 Sep 2023.

## Builds on (most cited first, 30 of 38)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose](https://mlsanity.com/paper/1803.02276/) (2018)
- [Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video](https://mlsanity.com/paper/1908.10553/) (2019)
- [FlowNet3D: Learning Scene Flow in 3D Point Clouds](https://mlsanity.com/paper/1806.01411/) (2018)
- [UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Loss](https://mlsanity.com/paper/1711.07837/) (2017)
- [DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistency](https://mlsanity.com/paper/1809.01649/) (2018)
- [FLOT: Scene Flow on Point Clouds Guided by Optimal Transport](https://mlsanity.com/paper/2007.11142/) (2020)
- [RAFT-3D: Scene Flow using Rigid-Motion Embeddings](https://mlsanity.com/paper/2012.00726/) (2020)
- [HPLFlowNet: Hierarchical Permutohedral Lattice FlowNet for Scene Flow Estimation on Large-scale Point Clouds](https://mlsanity.com/paper/1906.05332/) (2019)
- [Every Pixel Counts ++: Joint Learning of Geometry and Motion with 3D Holistic Understanding](https://mlsanity.com/paper/1810.06125/) (2018)
- [PV-RAFT: Point-Voxel Correlation Fields for Scene Flow Estimation of Point Clouds](https://mlsanity.com/paper/2012.00987/) (2020)
- [SMURF: Self-Teaching Multi-Frame Unsupervised RAFT with Full-Image Warping](https://mlsanity.com/paper/2105.07014/) (2021)
- [Self-Supervised Monocular Depth Estimation with Internal Feature Fusion](https://mlsanity.com/paper/2110.09482/) (2021)
- [Self-Supervised Monocular Scene Flow Estimation](https://mlsanity.com/paper/2004.04143/) (2020)
- [FlowNet3D++: Geometric Losses For Deep Scene Flow Estimation](https://mlsanity.com/paper/1912.01438/) (2019)
- [Deep Rigid Instance Scene Flow](https://mlsanity.com/paper/1904.08913/) (2019)
- [Forget About the LiDAR: Self-Supervised Depth Estimators with MED Probability Volumes](https://mlsanity.com/paper/2008.03633/) (2020)
- [Tangent Space Backpropagation for 3D Transformation Groups](https://mlsanity.com/paper/2103.12032/) (2021)
- [Learning Monocular Visual Odometry via Self-Supervised Long-Term Modeling](https://mlsanity.com/paper/2007.10983/) (2020)
- [Learning Rigidity in Dynamic Scenes with a Moving Camera for 3D Motion Field Estimation](https://mlsanity.com/paper/1804.04259/) (2018)
- [SENSE: a Shared Encoder Network for Scene-flow Estimation](https://mlsanity.com/paper/1910.12361/) (2019)
- [Bi-PointFlowNet: Bidirectional Learning for Point Cloud Based Scene Flow Estimation](https://mlsanity.com/paper/2207.07522/) (2022)
- [PLADE-Net: Towards Pixel-Level Accuracy for Self-Supervised Single-View Depth Estimation with Neural Positional Encoding and Distilled Matting Loss](https://mlsanity.com/paper/2103.07362/) (2021)
- [What Matters for 3D Scene Flow Network](https://mlsanity.com/paper/2207.09143/) (2022)
- [Self-Supervised Multi-Frame Monocular Scene Flow](https://mlsanity.com/paper/2105.02216/) (2021)
- [Mono-SF: Multi-View Geometry Meets Single-View Depth for Monocular Scene Flow Estimation of Dynamic Traffic Scenes](https://mlsanity.com/paper/1908.06316/) (2019)

## Built on by (most cited first, 4 of 4)

- [Zero-Shot Monocular Scene Flow Estimation in the Wild](https://mlsanity.com/paper/2501.10357/) (2025)
- [DELTAv2: Accelerating Dense 3D Tracking](https://mlsanity.com/paper/2508.01170/) (2025)
- [MAGIC: Learning from Visibility Asymmetry for Unsupervised Stereo Matching](https://mlsanity.com/paper/2508.10838/) (2025)
- [RAFT-MSF++: Temporal Geometry-Motion Feature Fusion for Self-Supervised Monocular Scene Flow](https://mlsanity.com/paper/2604.19349/) (2026)

## Abstract (arXiv)

Self-supervised monocular scene flow estimation, aiming to understand both 3D structures and 3D motions from two temporally consecutive monocular images, has received increasing attention for its simple and economical sensor setup. However, the accuracy of current methods suffers from the bottleneck of less-efficient network architecture and lack of motion rigidity for regularization. In this paper, we propose a superior model named EMR-MSF by borrowing the advantages of network architecture design under the scope of supervised learning. We further impose explicit and robust geometric constraints with an elaborately constructed ego-motion aggregation module where a rigidity soft mask is proposed to filter out dynamic regions for stable ego-motion estimation using static regions. Moreover, we propose a motion consistency loss along with a mask regularization loss to fully exploit static regions. Several efficient training strategies are integrated including a gradient detachment technique and an enhanced view synthesis process for better performance. Our proposed method outperforms the previous self-supervised works by a large margin and catches up to the performance of supervised methods. On the KITTI scene flow benchmark, our approach improves the SF-all metric of the state-of-the-art self-supervised monocular method by 44% and demonstrates superior performance across sub-tasks including depth and visual odometry, amongst other self-supervised single-task or multi-task methods.