# Out-of-boundary View Synthesis Towards Full-Frame Video Stabilization

arXiv: https://arxiv.org/abs/2108.09041 · HTML: https://mlsanity.com/paper/2108.09041/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 47 references that are in the map.
- First posted on arXiv 20 Aug 2021.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [Fully-Convolutional Siamese Networks for Object Tracking](https://mlsanity.com/paper/1606.09549/) (2016)
- [Video Object Segmentation using Space-Time Memory Networks](https://mlsanity.com/paper/1904.00607/) (2019)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [LIFT: Learned Invariant Feature Transform](https://mlsanity.com/paper/1603.09114/) (2016)
- [Depth-Aware Video Frame Interpolation](https://mlsanity.com/paper/1904.00830/) (2019)
- [Learning Joint Spatial-Temporal Transformations for Video Inpainting](https://mlsanity.com/paper/2007.10247/) (2020)
- [Deep Image Homography Estimation](https://mlsanity.com/paper/1606.03798/) (2016)
- [Learning Depth with Convolutional Spatial Propagation Network](https://mlsanity.com/paper/1810.02695/) (2018)
- [Empowering Things with Intelligence: A Survey of the Progress, Challenges, and Opportunities in Artificial Intelligence of Things](https://mlsanity.com/paper/2011.08612/) (2020)
- [Deep Video Inpainting](https://mlsanity.com/paper/1905.01639/) (2019)
- [Learning Affinity via Spatial Propagation Networks](https://mlsanity.com/paper/1710.01020/) (2017)
- [Unsupervised Deep Homography: A Fast and Robust Homography Estimation Model](https://mlsanity.com/paper/1709.03966/) (2017)
- [DeepSFM: Structure From Motion Via Deep Bundle Adjustment](https://mlsanity.com/paper/1912.09697/) (2019)
- [Content-Aware Unsupervised Deep Homography Estimation](https://mlsanity.com/paper/1909.05983/) (2019)
- [Fast Spatio-Temporal Residual Network for Video Super-Resolution](https://mlsanity.com/paper/1904.02870/) (2019)
- [On Exploring Undetermined Relationships for Visual Relationship Detection](https://mlsanity.com/paper/1905.01595/) (2019)
- [Deep Iterative Frame Interpolation for Full-frame Video Stabilization](https://mlsanity.com/paper/1909.02641/) (2019)
- [Towards High Performance Human Keypoint Detection](https://mlsanity.com/paper/2002.00537/) (2020)
- [DUT: Learning Video Stabilization by Simply Watching Unstable Videos](https://mlsanity.com/paper/2011.14574/) (2020)

## Built on by (most cited first, 5 of 5)

- [Fast Full-frame Video Stabilization with Iterative Optimization](https://mlsanity.com/paper/2307.12774/) (2023)
- [GlobalFlowNet: Video Stabilization using Deep Distilled Global Motion Estimates](https://mlsanity.com/paper/2210.13769/) (2022)
- [Harnessing Meta-Learning for Improving Full-Frame Video Stabilization](https://mlsanity.com/paper/2403.03662/) (2024)
- [Deep representation learning: Fundamentals, Perspectives, Applications, and Open Challenges](https://mlsanity.com/paper/2211.14732/) (2022)
- [Harnessing Meta-Learning for Controllable Full-Frame Video Stabilization](https://mlsanity.com/paper/2508.18859/) (2025)

## Abstract (arXiv)

Warping-based video stabilizers smooth camera trajectory by constraining each pixel's displacement and warp stabilized frames from unstable ones accordingly. However, since the view outside the boundary is not available during warping, the resulting holes around the boundary of the stabilized frame must be discarded (i.e., cropping) to maintain visual consistency, and thus does leads to a tradeoff between stability and cropping ratio. In this paper, we make a first attempt to address this issue by proposing a new Out-of-boundary View Synthesis (OVS) method. By the nature of spatial coherence between adjacent frames and within each frame, OVS extrapolates the out-of-boundary view by aligning adjacent frames to each reference one. Technically, it first calculates the optical flow and propagates it to the outer boundary region according to the affinity, and then warps pixels accordingly. OVS can be integrated into existing warping-based stabilizers as a plug-and-play module to significantly improve the cropping ratio of the stabilized results. In addition, stability is improved because the jitter amplification effect caused by cropping and resizing is reduced. Experimental results on the NUS benchmark show that OVS can improve the performance of five representative state-of-the-art methods in terms of objective metrics and subjective visual quality. The code is publicly available at https://github.com/Annbless/OVS_Stabilization.