# GlobalFlowNet: Video Stabilization using Deep Distilled Global Motion Estimates

arXiv: https://arxiv.org/abs/2210.13769 · HTML: https://mlsanity.com/paper/2210.13769/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 43 references that are in the map.
- First posted on arXiv 25 Oct 2022.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Semantic Understanding of Scenes through the ADE20K Dataset](https://mlsanity.com/paper/1608.05442/) (2016)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [A General and Adaptive Robust Loss Function](https://mlsanity.com/paper/1701.03077/) (2017)
- [Stereo Magnification: Learning View Synthesis using Multiplane Images](https://mlsanity.com/paper/1805.09817/) (2018)
- [DDFlow: Learning Optical Flow with Unlabeled Data Distillation](https://mlsanity.com/paper/1902.09145/) (2019)
- [Deep Iterative Frame Interpolation for Full-frame Video Stabilization](https://mlsanity.com/paper/1909.02641/) (2019)
- [Hybrid Neural Fusion for Full-frame Video Stabilization](https://mlsanity.com/paper/2102.06205/) (2021)
- [Learning to Solve Hard Minimal Problems](https://mlsanity.com/paper/2112.03424/) (2021)
- [Deep Online Fused Video Stabilization](https://mlsanity.com/paper/2102.01279/) (2021)
- [Real-Time Optical flow-based Video Stabilization for Unmanned Aerial Vehicles](https://mlsanity.com/paper/1701.03572/) (2017)
- [Out-of-boundary View Synthesis Towards Full-Frame Video Stabilization](https://mlsanity.com/paper/2108.09041/) (2021)
- [Deep Online Video Stabilization](https://mlsanity.com/paper/1802.08091/) (2018)

## Built on by (most cited first, 7 of 7)

- [StableVQA: A Deep No-Reference Quality Assessment Model for Video Stability](https://mlsanity.com/paper/2308.04904/) (2023)
- [NTIRE 2023 Quality Assessment of Video Enhancement Challenge](https://mlsanity.com/paper/2307.09729/) (2023)
- [Harnessing Meta-Learning for Improving Full-Frame Video Stabilization](https://mlsanity.com/paper/2403.03662/) (2024)
- [RMT-BVQA: Recurrent Memory Transformer-based Blind Video Quality Assessment for Enhanced Video Content](https://mlsanity.com/paper/2405.08621/) (2024)
- [ToonifyGB: StyleGAN-based Gaussian Blendshapes for 3D Stylized Head Avatars](https://mlsanity.com/paper/2505.10072/) (2025)
- [Harnessing Meta-Learning for Controllable Full-Frame Video Stabilization](https://mlsanity.com/paper/2508.18859/) (2025)
- [Beyond Wide-Angle Images: Structure-to-Detail Video Portrait Correction via Unsupervised Spatiotemporal Adaptation](https://mlsanity.com/paper/2504.00401/) (2025)

## Abstract (arXiv)

Videos shot by laymen using hand-held cameras contain undesirable shaky motion. Estimating the global motion between successive frames, in a manner not influenced by moving objects, is central to many video stabilization techniques, but poses significant challenges. A large body of work uses 2D affine transformations or homography for the global motion. However, in this work, we introduce a more general representation scheme, which adapts any existing optical flow network to ignore the moving objects and obtain a spatially smooth approximation of the global motion between video frames. We achieve this by a knowledge distillation approach, where we first introduce a low pass filter module into the optical flow network to constrain the predicted optical flow to be spatially smooth. This becomes our student network, named as textscGlobalFlowNet. Then, using the original optical flow network as the teacher network, we train the student network using a robust loss function. Given a trained textscGlobalFlowNet, we stabilize videos using a two stage process. In the first stage, we correct the instability in affine parameters using a quadratic programming approach constrained by a user-specified cropping limit to control loss of field of view. In the second stage, we stabilize the video further by smoothing global motion parameters, expressed using a small number of discrete cosine transform coefficients. In extensive experiments on a variety of different videos, our technique outperforms state of the art techniques in terms of subjective quality and different quantitative measures of video stability. The source code is publicly available at hrefhttps://github.com/GlobalFlowNet/GlobalFlowNethttps://github.com/GlobalFlowNet/GlobalFlowNet