# GyroFlow: Gyroscope-Guided Unsupervised Optical Flow Learning

arXiv: https://arxiv.org/abs/2103.13725 · HTML: https://mlsanity.com/paper/2103.13725/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 54 references that are in the map.
- First posted on arXiv 25 Mar 2021.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [Optical Flow Estimation using a Spatial Pyramid Network](https://mlsanity.com/paper/1611.00850/) (2016)
- [GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose](https://mlsanity.com/paper/1803.02276/) (2018)
- [UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Loss](https://mlsanity.com/paper/1711.07837/) (2017)
- [Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation](https://mlsanity.com/paper/1904.05290/) (2019)
- [SelFlow: Self-Supervised Learning of Optical Flow](https://mlsanity.com/paper/1904.09117/) (2019)
- [What Matters in Unsupervised Optical Flow](https://mlsanity.com/paper/2006.04902/) (2020)
- [Learning by Analogy: Reliable Supervision from Transformations for Unsupervised Optical Flow Estimation](https://mlsanity.com/paper/2003.13045/) (2020)
- [Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothness](https://mlsanity.com/paper/1608.05842/) (2016)
- [Occlusion Aware Unsupervised Learning of Optical Flow](https://mlsanity.com/paper/1711.05890/) (2017)
- [Fast Optical Flow using Dense Inverse Search](https://mlsanity.com/paper/1603.03590/) (2016)
- [DDFlow: Learning Optical Flow with Unlabeled Data Distillation](https://mlsanity.com/paper/1902.09145/) (2019)
- [UPFlow: Upsampling Pyramid for Unsupervised Optical Flow Learning](https://mlsanity.com/paper/2012.00212/) (2020)
- [Unsupervised Deep Epipolar Flow for Stationary or Dynamic Scenes](https://mlsanity.com/paper/1904.03848/) (2019)
- [Optical Flow in Dense Foggy Scenes using Semi-Supervised Learning](https://mlsanity.com/paper/2004.01905/) (2020)
- [Gyroscope-Aided Motion Deblurring with Deep Networks](https://mlsanity.com/paper/1810.00986/) (2018)
- [DeepOIS: Gyroscope-Guided Deep Optical Image Stabilizer Compensation](https://mlsanity.com/paper/2101.11183/) (2021)

## Built on by (most cited first, 15 of 15)

- [RealFlow: EM-based Realistic Optical Flow Dataset Generation from Videos](https://mlsanity.com/paper/2207.11075/) (2022)
- [Deep Learning for Camera Calibration and Beyond: A Survey](https://mlsanity.com/paper/2303.10559/) (2023)
- [MDFlow: Unsupervised Optical Flow Learning by Reliable Mutual Knowledge Distillation](https://mlsanity.com/paper/2211.06018/) (2022)
- [Semi-supervised Deep Large-baseline Homography Estimation with Progressive Equivalence Constraint](https://mlsanity.com/paper/2212.02763/) (2022)
- [GyroFlow+: Gyroscope-Guided Unsupervised Deep Homography and Optical Flow Learning](https://mlsanity.com/paper/2301.10018/) (2023)
- [Unsupervised Cumulative Domain Adaptation for Foggy Scene Optical Flow](https://mlsanity.com/paper/2303.07564/) (2023)
- [Supervised Homography Learning with Realistic Dataset Generation](https://mlsanity.com/paper/2307.15353/) (2023)
- [Single Image Rolling Shutter Removal with Diffusion Models](https://mlsanity.com/paper/2407.02906/) (2024)
- [Adverse Weather Optical Flow: Cumulative Homogeneous-Heterogeneous Adaptation](https://mlsanity.com/paper/2409.17001/) (2024)
- [Unsupervised Hierarchical Domain Adaptation for Adverse Weather Optical Flow](https://mlsanity.com/paper/2303.13761/) (2023)
- [CodingHomo: Bootstrapping Deep Homography With Video Coding](https://mlsanity.com/paper/2504.12165/) (2025)
- [Toward Efficient Visual Gyroscopes: Spherical Moments, Harmonics Filtering, and Masking Techniques for Spherical Camera Applications](https://mlsanity.com/paper/2404.01924/) (2024)
- [Estimating 2D Camera Motion with Hybrid Motion Basis](https://mlsanity.com/paper/2507.22480/) (2025)
- [CamFlow+: Hybrid Motion Bases for 2D Camera Motion Estimation with Stabilization Applications](https://mlsanity.com/paper/2606.05915/) (2026)
- [On the Real-World Generalisability of Optical Flow Models](https://mlsanity.com/paper/2607.10470/) (2026)

## Abstract (arXiv)

Existing optical flow methods are erroneous in challenging scenes, such as fog, rain, and night because the basic optical flow assumptions such as brightness and gradient constancy are broken. To address this problem, we present an unsupervised learning approach that fuses gyroscope into optical flow learning. Specifically, we first convert gyroscope readings into motion fields named gyro field. Second, we design a self-guided fusion module to fuse the background motion extracted from the gyro field with the optical flow and guide the network to focus on motion details. To the best of our knowledge, this is the first deep learning-based framework that fuses gyroscope data and image content for optical flow learning. To validate our method, we propose a new dataset that covers regular and challenging scenes. Experiments show that our method outperforms the state-of-art methods in both regular and challenging scenes. Code and dataset are available at https://github.com/megvii-research/GyroFlow.