# Dimensions of Motion: Monocular Prediction through Flow Subspaces

arXiv: https://arxiv.org/abs/2112.01502 · HTML: https://mlsanity.com/paper/2112.01502/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 58 references that are in the map.
- First posted on arXiv 2 Dec 2021.

## Builds on (most cited first, 30 of 34)

- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue](https://mlsanity.com/paper/1603.04992/) (2016)
- [Single-View View Synthesis with Multiplane Images](https://mlsanity.com/paper/2004.11364/) (2020)
- [Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video](https://mlsanity.com/paper/1908.10553/) (2019)
- [Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentation](https://mlsanity.com/paper/1805.09806/) (2018)
- [Associative Embedding: End-to-End Learning for Joint Detection and Grouping](https://mlsanity.com/paper/1611.05424/) (2016)
- [Stereo Magnification: Learning View Synthesis using Multiplane Images](https://mlsanity.com/paper/1805.09817/) (2018)
- [Single-Image Depth Perception in the Wild](https://mlsanity.com/paper/1604.03901/) (2016)
- [Evaluation of CNN-based Single-Image Depth Estimation Methods](https://mlsanity.com/paper/1805.01328/) (2018)
- [MINE: Towards Continuous Depth MPI with NeRF for Novel View Synthesis](https://mlsanity.com/paper/2103.14910/) (2021)
- [3D Ken Burns Effect from a Single Image](https://mlsanity.com/paper/1909.05483/) (2019)
- [RAFT-3D: Scene Flow using Rigid-Motion Embeddings](https://mlsanity.com/paper/2012.00726/) (2020)
- [Learning the Depths of Moving People by Watching Frozen People](https://mlsanity.com/paper/1904.11111/) (2019)
- [SfM-Net: Learning of Structure and Motion from Video](https://mlsanity.com/paper/1704.07804/) (2017)
- [Towards Better Generalization: Joint Depth-Pose Learning without PoseNet](https://mlsanity.com/paper/2004.01314/) (2020)
- [A Survey on Instance Segmentation: State of the art](https://mlsanity.com/paper/2007.00047/) (2020)
- [Unsupervised Monocular Depth Learning in Dynamic Scenes](https://mlsanity.com/paper/2010.16404/) (2020)
- [Animating Pictures with Eulerian Motion Fields](https://mlsanity.com/paper/2011.15128/) (2020)
- [Semantic Instance Segmentation via Deep Metric Learning](https://mlsanity.com/paper/1703.10277/) (2017)
- [Dense Optical Flow Prediction from a Static Image](https://mlsanity.com/paper/1505.00295/) (2015)
- [Multiway Non-rigid Point Cloud Registration via Learned Functional Map Synchronization](https://mlsanity.com/paper/2111.12878/) (2021)
- [Im2Flow: Motion Hallucination from Static Images for Action Recognition](https://mlsanity.com/paper/1712.04109/) (2017)

## Built on by (most cited first, 10 of 10)

- [PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generation](https://mlsanity.com/paper/2409.18964/) (2024)
- [Generative Image Dynamics](https://mlsanity.com/paper/2309.07906/) (2023)
- [PhysGen3D: Crafting a Miniature Interactive World from a Single Image](https://mlsanity.com/paper/2503.20746/) (2025)
- [Diffusion Models for Open-Vocabulary Segmentation](https://mlsanity.com/paper/2306.09316/) (2023)
- [FlowMap: High-Quality Camera Poses, Intrinsics, and Depth via Gradient Descent](https://mlsanity.com/paper/2404.15259/) (2024)
- [Multi-Object Discovery by Low-Dimensional Object Motion](https://mlsanity.com/paper/2307.08027/) (2023)
- [FlowSeek: Optical Flow Made Easier with Depth Foundation Models and Motion Bases](https://mlsanity.com/paper/2509.05297/) (2025)
- [Discriminately Treating Motion Components Evolves Joint Depth and Ego-Motion Learning](https://mlsanity.com/paper/2511.01502/) (2025)
- [PixRO: Pixel-Distributed Rotational Odometry with Gaussian Belief Propagation](https://mlsanity.com/paper/2406.09726/) (2024)
- [SIRE: SE(3) Intrinsic Rigidity Embeddings](https://mlsanity.com/paper/2503.07739/) (2025)

## Abstract (arXiv)

We introduce a way to learn to estimate a scene representation from a single image by predicting a low-dimensional subspace of optical flow for each training example, which encompasses the variety of possible camera and object movement. Supervision is provided by a novel loss which measures the distance between this predicted flow subspace and an observed optical flow. This provides a new approach to learning scene representation tasks, such as monocular depth prediction or instance segmentation, in an unsupervised fashion using in-the-wild input videos without requiring camera poses, intrinsics, or an explicit multi-view stereo step. We evaluate our method in multiple settings, including an indoor depth prediction task where it achieves comparable performance to recent methods trained with more supervision.