mlsanityData to 14 Oct 2026

Self-Supervised Multi-Frame Monocular Scene Flow

arXiv 2105.02216 ↗First posted 5 May 2021cs.CV cs.LG cs.ROcited by 29

Not on the map. Why?

Abstract (arXiv)

Estimating 3D scene flow from a sequence of monocular images has been gaining increased attention due to the simple, economical capture setup. Owing to the severe ill-posedness of the problem, the accuracy of current methods has been limited, especially that of efficient, real-time approaches. In this paper, we introduce a multi-frame monocular scene flow network based on self-supervised learning, improving the accuracy over previous networks while retaining real-time efficiency. Based on an advanced two-frame baseline with a split-decoder design, we propose (i) a multi-frame model using a triple frame input and convolutional LSTM connections, (ii) an occlusion-aware census loss for better accuracy, and (iii) a gradient detaching strategy to improve training stability. On the KITTI dataset, we observe state-of-the-art accuracy among monocular scene flow methods based on self-supervised learning.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Builds on 44 of its 85 references that are in the map. First posted on arXiv 5 May 2021.

Builds on

44 of 85 references.

References · 44

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  3. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,436
  4. 2015Spatial Transformer Networkscited by 1,429
  5. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 968
  6. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 881
  7. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 820
  8. 2018Open3D: A Modern Library for 3D Data Processingcited by 721
  9. 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 342
  10. 2020Softmax Splatting for Video Frame Interpolationcited by 279
  11. 2018FlowNet3D: Learning Scene Flow in 3D Point Cloudscited by 242
  12. 2018Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentationcited by 222
32 more
  1. 2017UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Losscited by 202
  2. 2018DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistencycited by 148
  3. 2019Self-supervised Learning with Geometric Constraints in Monocular Video: Connecting Flow, Depth, and Cameracited by 130
  4. 2019SelFlow: Self-Supervised Learning of Optical Flowcited by 128
  5. 2020What Matters in Unsupervised Optical Flowcited by 124
  6. 2019HPLFlowNet: Hierarchical Permutohedral Lattice FlowNet for Scene Flow Estimation on Large-scale Point Cloudscited by 109
  7. 2018Every Pixel Counts ++: Joint Learning of Geometry and Motion with 3D Holistic Understandingcited by 101
  8. 2019Just Go with the Flow: Self-Supervised Scene Flow Estimationcited by 98
  9. 2019Web Stereo Video Supervision for Depth Prediction from Dynamic Scenescited by 73
  10. 2018Occlusions, Motion and Depth Boundaries with a Generic Network for Disparity, Optical Flow or Scene Flow Estimationcited by 69
  11. 2020Self-Supervised Monocular Scene Flow Estimationcited by 64
  12. 2019DDFlow: Learning Optical Flow with Unlabeled Data Distillationcited by 63
  13. 2019FlowNet3D++: Geometric Losses For Deep Scene Flow Estimationcited by 55
  14. 2019Deep Rigid Instance Scene Flowcited by 54
  15. 2018Learning Rigidity in Dynamic Scenes with a Moving Camera for 3D Motion Field Estimationcited by 38
  16. 2019SENSE: a Shared Encoder Network for Scene-flow Estimationcited by 38
  17. 2018A Fusion Approach for Multi-Frame Optical Flow Estimationcited by 37
  18. 2019Mono-SF: Multi-View Geometry Meets Single-View Depth for Monocular Scene Flow Estimation of Dynamic Traffic Scenescited by 27
  19. 2018Every Pixel Counts: Unsupervised Geometry Learning with Holistic 3D Motion Understandingcited by 24
  20. 2019Bridging Stereo Matching and Optical Flow via Spatiotemporal Correspondencecited by 23
  21. 2019PWOC-3D: Deep Occlusion-Aware End-to-End Scene Flow Estimationcited by 20
  22. 2019Learning End-To-End Scene Flow by Distilling Single Tasks Knowledgecited by 16
  23. 2019Learning Residual Flow as Dynamic Motion from Stereo Videoscited by 12
  24. 2020STaRFlow: A SpatioTemporal Recurrent Cell for Lightweight Multi-Frame Optical Flow Estimationcited by 12
  25. 2018ProFlow: Learning to Predict Optical Flowcited by 10
  26. 2017Fast Multi-frame Stereo Scene Flow with Motion Segmentationcited by 6
  27. 2017Multiframe Scene Flow with Piecewise Rigid Motioncited by 6
  28. 2017SceneFlowFields: Dense Interpolation of Sparse Scene Flow Correspondencescited by 6
  29. 2019SceneFlowFields++: Multi-frame Matching, Visibility Prediction, and Robust Interpolation for Scene Flow Estimationcited by 6
  30. 2018Robustness Meets Deep Learning: An End-to-End Hybrid Pipeline for Unsupervised Learning of Egomotioncited by 5
  31. 2019Unsupervised Learning for Optical Flow Estimation Using Pyramid Convolution LSTMcited by 3
  32. 2018Continual Occlusions and Optical Flow Estimationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 10; 2023: 10; 2024: 5; 2025: 2; 2026: 1.

29 papers.

Sort:

  1. 2023VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimationcited by 82
  2. 2022Multi-Frame Self-Supervised Depth with Transformerscited by 64
  3. 2022IFOR: Iterative Flow Minimization for Robotic Object Rearrangementcited by 42
  4. 2023Fast Neural Scene Flowcited by 41
  5. 2022Learning Optical Flow, Depth, and Scene Flow without Real-World Labelscited by 31
  6. 2022What Matters for 3D Scene Flow Networkcited by 31
  7. 2025Zero-Shot Monocular Scene Flow Estimation in the Wildcited by 22
  8. 2023Dynamo-Depth: Fixing Unsupervised Depth Estimation for Dynamical Scenescited by 20
  9. 2024Bring Event into RGB and LiDAR: Hierarchical Visual-Motion Fusion for Scene Flowcited by 15
  10. 2022M-FUSE: Multi-frame Fusion for Scene Flow Estimationcited by 14
  11. 2023Learning Optical Flow and Scene Flow with Bidirectional Camera-LiDAR Fusioncited by 10
  12. 2025DELTAv2: Accelerating Dense 3D Trackingcited by 10
17 more
  1. 2023ADFactory: An Effective Framework for Generalizing Optical Flow with Nerfcited by 9
  2. 2023DS-Depth: Dynamic and Static Depth Estimation via a Fusion Cost Volumecited by 8
  3. 2022RAFT-MSF: Self-Supervised Monocular Scene Flow using Recurrent Optimizercited by 7
  4. 20223D Scene Flow Estimation on Pseudo-LiDAR: Bridging the Gap on Estimating Point Motioncited by 7
  5. 2023Learning to Estimate Single-View Volumetric Flow Motions without 3D Supervisioncited by 5
  6. 2023EMR-MSF: Self-Supervised Recurrent Monocular Scene Flow Exploiting Ego-Motion Rigiditycited by 4
  7. 2022Time-to-Label: Temporal Consistency for Self-Supervised Monocular 3D Object Detectioncited by 3
  8. 2022Learning Scene Flow in 3D Point Clouds with Noisy Pseudo Labelscited by 3
  9. 2023ContrastMotion: Self-supervised Scene Motion Learning for Large-Scale LiDAR Point Cloudscited by 3
  10. 2024Self-Supervised Multi-Frame Neural Scene Flowcited by 3
  11. 2024Self-Assessed Generation: Trustworthy Label Generation for Optical Flow and Stereo Matching in Real-worldcited by 3
  12. 2021MonoPLFlowNet: Permutohedral Lattice FlowNet for Real-Scale 3D Scene FlowEstimation with Monocular Imagescited by 2
  13. 2023DEFLOW: Self-supervised 3D Motion Estimation of Debris Flowcited by 1
  14. 2024Human Insights Driven Latent Space for Different Driving Perspectives: A Unified Encoder for Efficient Multi-Task Inferencecited by 1
  15. 2022Self-SuperFlow: Self-supervised Scene Flow Prediction in Stereo Sequencescited by 0
  16. 2024Pseudo-Stereo Inputs: A Solution to the Occlusion Challenge in Self-Supervised Stereo Matchingcited by 0
  17. 2026RAFT-MSF++: Temporal Geometry-Motion Feature Fusion for Self-Supervised Monocular Scene Flowcited by 0