mlsanityData to 14 Oct 2026

Dimensions of Motion: Monocular Prediction through Flow Subspaces

arXiv 2112.01502 ↗First posted 2 Dec 2021cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

We introduce a way to learn to estimate a scene representation from a single image by predicting a low-dimensional subspace of optical flow for each training example, which encompasses the variety of possible camera and object movement. Supervision is provided by a novel loss which measures the distance between this predicted flow subspace and an observed optical flow. This provides a new approach to learning scene representation tasks, such as monocular depth prediction or instance segmentation, in an unsupervised fashion using in-the-wild input videos without requiring camera poses, intrinsics, or an explicit multi-view stereo step. We evaluate our method in multiple settings, including an indoor depth prediction task where it achieves comparable performance to recent methods trained with more supervision.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 34 of its 58 references that are in the map. First posted on arXiv 2 Dec 2021.

Builds on

34 of 58 references.

References · 34

  1. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  2. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  3. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  4. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 968
  5. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 881
  6. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 820
  7. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 782
  8. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 737
  9. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 723
  10. 2016Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescuecited by 368
  11. 2020Single-View View Synthesis with Multiplane Imagescited by 263
  12. 2019Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Videocited by 244
22 more
  1. 2018Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentationcited by 222
  2. 2016Associative Embedding: End-to-End Learning for Joint Detection and Groupingcited by 212
  3. 2018Stereo Magnification: Learning View Synthesis using Multiplane Imagescited by 211
  4. 2016Single-Image Depth Perception in the Wildcited by 180
  5. 2018Evaluation of CNN-based Single-Image Depth Estimation Methodscited by 138
  6. 2021MINE: Towards Continuous Depth MPI with NeRF for Novel View Synthesiscited by 133
  7. 20193D Ken Burns Effect from a Single Imagecited by 128
  8. 2020RAFT-3D: Scene Flow using Rigid-Motion Embeddingscited by 122
  9. 2019Learning the Depths of Moving People by Watching Frozen Peoplecited by 120
  10. 2017SfM-Net: Learning of Structure and Motion from Videocited by 89
  11. 2020Towards Better Generalization: Joint Depth-Pose Learning without PoseNetcited by 85
  12. 2020A Survey on Instance Segmentation: State of the artcited by 85
  13. 2020Unsupervised Monocular Depth Learning in Dynamic Scenescited by 84
  14. 2020Animating Pictures with Eulerian Motion Fieldscited by 55
  15. 2017Semantic Instance Segmentation via Deep Metric Learningcited by 37
  16. 2015Dense Optical Flow Prediction from a Static Imagecited by 36
  17. 2021Multiway Non-rigid Point Cloud Registration via Learned Functional Map Synchronizationcited by 30
  18. 2017Im2Flow: Motion Hallucination from Static Images for Action Recognitioncited by 24
  19. 2017Optical Flow in Mostly Rigid Scenescited by 20
  20. 2018Visual Dynamics: Stochastic Future Generation via Layered Cross Convolutional Networkscited by 11
  21. 2018Deja Vu: Motion Prediction in Static Imagescited by 7
  22. 2021Revisiting Self-Supervised Monocular Depth Estimationcited by 2

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 3; 2025: 4.

10 papers.

  1. 2024PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generationcited by 174
  2. 2023Generative Image Dynamicscited by 94
  3. 2025PhysGen3D: Crafting a Miniature Interactive World from a Single Imagecited by 66
  4. 2023Diffusion Models for Open-Vocabulary Segmentationcited by 64
  5. 2024FlowMap: High-Quality Camera Poses, Intrinsics, and Depth via Gradient Descentcited by 61
  6. 2023Multi-Object Discovery by Low-Dimensional Object Motioncited by 16
  7. 2025FlowSeek: Optical Flow Made Easier with Depth Foundation Models and Motion Basescited by 10
  8. 2025Discriminately Treating Motion Components Evolves Joint Depth and Ego-Motion Learningcited by 1
  9. 2024PixRO: Pixel-Distributed Rotational Odometry with Gaussian Belief Propagationcited by 0
  10. 2025SIRE: SE(3) Intrinsic Rigidity Embeddingscited by 0