mlsanityData to 14 Oct 2026

ScaleFlow++: Robust and Accurate Estimation of 3D Motion from Video

arXiv 2409.12202 ↗First posted 16 Sep 2024cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Perceiving and understanding 3D motion is a core technology in fields such as autonomous driving, robots, and motion prediction. This paper proposes a 3D motion perception method called ScaleFlow++ that is easy to generalize. With just a pair of RGB images, ScaleFlow++ can robustly estimate optical flow and motion-in-depth (MID). Most existing methods directly regress MID from two RGB frames or optical flow, resulting in inaccurate and unstable results. Our key insight is cross-scale matching, which extracts deep motion clues by matching objects in pairs of images at different scales. Unlike previous methods, ScaleFlow++ integrates optical flow and MID estimation into a unified architecture, estimating optical flow and MID end-to-end based on feature matching. Moreover, we also proposed modules such as global initialization network, global iterative optimizer, and hybrid training pipeline to integrate global motion information, reduce the number of iterations, and prevent overfitting during training. On KITTI, ScaleFlow++ achieved the best monocular scene flow estimation performance, reducing SF-all from 6.21 to 5.79. The evaluation of MID even surpasses RGBD-based methods. In addition, ScaleFlow++ has achieved stunning zero-shot generalization performance in both rigid and nonrigid scenes. Code is available at urlhttps://github.com/HanLingsgjk/CSCV.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 42 of its 71 references that are in the map. First posted on arXiv 16 Sep 2024.

Builds on

42 of 71 references.

References · 42

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  3. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  4. 2013Visualizing and Understanding Convolutional Networkscited by 2,290
  5. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,059
  6. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 968
  7. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 881
  8. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 784
  9. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 721
  10. 2014Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognitioncited by 720
  11. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 712
  12. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 506
30 more
  1. 2016Optical Flow Estimation using a Spatial Pyramid Networkcited by 424
  2. 2020UNet 3+: A Full-Scale Connected UNet for Medical Image Segmentationcited by 332
  3. 2021GMFlow: Learning Optical Flow via Global Matchingcited by 327
  4. 2022FlowFormer: A Transformer Architecture for Optical Flowcited by 275
  5. 2022Unifying Flow, Stereo and Depth Estimationcited by 275
  6. 2019GA-Net: Guided Aggregation Net for End-to-end Stereo Matchingcited by 244
  7. 2018FlowNet3D: Learning Scene Flow in 3D Point Cloudscited by 242
  8. 2018Human Action Recognition and Prediction: A Surveycited by 227
  9. 2021Learning to Estimate Hidden Motions with Global Motion Aggregationcited by 224
  10. 2020Consistent Video Depth Estimationcited by 220
  11. 2023Iterative Geometry Encoding Volume for Stereo Matchingcited by 190
  12. 2024SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flowcited by 174
  13. 2020FLOT: Scene Flow on Point Clouds Guided by Optimal Transportcited by 129
  14. 2020RAFT-3D: Scene Flow using Rigid-Motion Embeddingscited by 122
  15. 2021MSeg: A Composite Dataset for Multi-domain Semantic Segmentationcited by 122
  16. 2022CRAFT: Cross-Attentional Flow Transformer for Robust Optical Flowcited by 82
  17. 2020PV-RAFT: Point-Voxel Correlation Fields for Scene Flow Estimation of Point Cloudscited by 74
  18. 2020Self-Supervised Monocular Scene Flow Estimationcited by 64
  19. 2022Global Matching with Overlapping Attention for Optical Flow Estimationcited by 64
  20. 2021CamLiFlow: Bidirectional Camera-LiDAR Fusion for Joint Optical Flow and Scene Flow Estimationcited by 56
  21. 2021Learning to Segment Rigid Motions from Two Framescited by 46
  22. 2021Real-Time Optical Flow for Vehicular Perception with Low- and High-Resolution Event Camerascited by 27
  23. 2022PanoFlow: Learning 360deg Optical Flow for Surrounding Temporal Understandingcited by 26
  24. 2021Binary TTC: A Temporal Geofence for Autonomous Navigationcited by 20
  25. 2018Devon: Deformable Volume Network for Learning Optical Flowcited by 16
  26. 2022High Resolution Multi-Scale RAFT (Robust Vision Challenge 2022)cited by 12
  27. 2023ADFactory: An Effective Framework for Generalizing Optical Flow with Nerfcited by 9
  28. 2024Optical Flow Based Detection and Tracking of Moving Objects for Autonomous Vehiclescited by 6
  29. 2016AutoScaler: Scale-Attention Networks for Visual Correspondencecited by 5
  30. 2019Forecasting Time-to-Collision from Monocular Video: Feasibility, Dataset, and Challengescited by 5

Built on by

Not cited yet.