# DUT: Learning Video Stabilization by Simply Watching Unstable Videos

arXiv: https://arxiv.org/abs/2011.14574 · HTML: https://mlsanity.com/paper/2011.14574/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Nov 2020.

## Built on by (most cited first, 16 of 16)

- [StableVQA: A Deep No-Reference Quality Assessment Model for Video Stability](https://mlsanity.com/paper/2308.04904/) (2023)
- [Minimum Latency Deep Online Video Stabilization](https://mlsanity.com/paper/2212.02073/) (2022)
- [Fast Full-frame Video Stabilization with Iterative Optimization](https://mlsanity.com/paper/2307.12774/) (2023)
- [3D Multi-frame Fusion for Video Stabilization](https://mlsanity.com/paper/2404.12887/) (2024)
- [Out-of-boundary View Synthesis Towards Full-Frame Video Stabilization](https://mlsanity.com/paper/2108.09041/) (2021)
- [Harnessing Meta-Learning for Improving Full-Frame Video Stabilization](https://mlsanity.com/paper/2403.03662/) (2024)
- [GaVS: 3D-Grounded Video Stabilization via Temporally-Consistent Local Reconstruction and Rendering](https://mlsanity.com/paper/2506.23957/) (2025)
- [Eliminating Warping Shakes for Unsupervised Online Video Stitching](https://mlsanity.com/paper/2403.06378/) (2024)
- [StabStitch++: Unsupervised Online Video Stitching with Spatiotemporal Bidirectional Warps](https://mlsanity.com/paper/2505.05001/) (2025)
- [Harnessing Meta-Learning for Controllable Full-Frame Video Stabilization](https://mlsanity.com/paper/2508.18859/) (2025)
- [Leveraging Synthetic Data to Learn Video Stabilization Under Adverse Conditions](https://mlsanity.com/paper/2208.12763/) (2022)
- [On the Benefits of Visual Stabilization for Frame- and Event-based Perception](https://mlsanity.com/paper/2408.15602/) (2024)
- [Beyond Wide-Angle Images: Structure-to-Detail Video Portrait Correction via Unsupervised Spatiotemporal Adaptation](https://mlsanity.com/paper/2504.00401/) (2025)
- [No Labels, No Look-Ahead: Unsupervised Online Video Stabilization with Classical Priors](https://mlsanity.com/paper/2602.23141/) (2026)
- [VS3R: Robust Full-frame Video Stabilization via Deep 3D Reconstruction](https://mlsanity.com/paper/2603.05851/) (2026)
- [SA-Homo: Scale Adaptive Homography Estimation for Scale Variation Scenarios](https://mlsanity.com/paper/2606.30408/) (2026)

## Abstract (arXiv)

Previous deep learning-based video stabilizers require a large scale of paired unstable and stable videos for training, which are difficult to collect. Traditional trajectory-based stabilizers, on the other hand, divide the task into several sub-tasks and tackle them subsequently, which are fragile in textureless and occluded regions regarding the usage of hand-crafted features. In this paper, we attempt to tackle the video stabilization problem in a deep unsupervised learning manner, which borrows the divide-and-conquer idea from traditional stabilizers while leveraging the representation power of DNNs to handle the challenges in real-world scenarios. Technically, DUT is composed of a trajectory estimation stage and a trajectory smoothing stage. In the trajectory estimation stage, we first estimate the motion of keypoints, initialize and refine the motion of grids via a novel multi-homography estimation strategy and a motion refinement network, respectively, and get the grid-based trajectories via temporal association. In the trajectory smoothing stage, we devise a novel network to predict dynamic smoothing kernels for trajectory smoothing, which can well adapt to trajectories with different dynamic patterns. We exploit the spatial and temporal coherence of keypoints and grid vertices to formulate the training objectives, resulting in an unsupervised training scheme. Experiment results on public benchmarks show that DUT outperforms state-of-the-art methods both qualitatively and quantitatively. The source code is available at https://github.com/Annbless/DUTCode.