# Learning Adaptive Warping for Real-World Rolling Shutter Correction

arXiv: https://arxiv.org/abs/2204.13886 · HTML: https://mlsanity.com/paper/2204.13886/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 35 references that are in the map.
- First posted on arXiv 29 Apr 2022.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [CCNet: Criss-Cross Attention for Semantic Segmentation](https://mlsanity.com/paper/1811.11721/) (2018)
- [Stand-Alone Self-Attention in Vision Models](https://mlsanity.com/paper/1906.05909/) (2019)
- [Toward Real-World Single Image Super-Resolution: A New Benchmark and A New Model](https://mlsanity.com/paper/1904.00523/) (2019)
- [Learning Texture Transformer Network for Image Super-Resolution](https://mlsanity.com/paper/2006.04139/) (2020)
- [Rolling-Shutter-Aware Differential SfM and Image Rectification](https://mlsanity.com/paper/1903.03943/) (2019)
- [Towards Rolling Shutter Correction and Deblurring in Dynamic Scenes](https://mlsanity.com/paper/2104.01601/) (2021)
- [Rolling Shutter Camera Relative Pose: Generalized Epipolar Geometry](https://mlsanity.com/paper/1605.00475/) (2016)
- [From two rolling shutters to one global shutter](https://mlsanity.com/paper/2006.01964/) (2020)
- [SUNet: Symmetric Undistortion Network for Rolling Shutter Correction](https://mlsanity.com/paper/2108.04775/) (2021)

## Built on by (most cited first, 13 of 13)

- [Blur Interpolation Transformer for Real-World Motion from Blur](https://mlsanity.com/paper/2211.11423/) (2022)
- [Deep Learning for Camera Calibration and Beyond: A Survey](https://mlsanity.com/paper/2303.10559/) (2023)
- [CMTA: Cross-Modal Temporal Alignment for Event-guided Video Deblurring](https://mlsanity.com/paper/2408.14930/) (2024)
- [Towards Nonlinear-Motion-Aware and Occlusion-Robust Rolling Shutter Correction](https://mlsanity.com/paper/2303.18125/) (2023)
- [RS-NeRF: Neural Radiance Fields from Rolling Shutter Images](https://mlsanity.com/paper/2407.10267/) (2024)
- [Self-supervised Learning of Event-guided Video Frame Interpolation for Rolling Shutter Frames](https://mlsanity.com/paper/2306.15507/) (2023)
- [Single Image Rolling Shutter Removal with Diffusion Models](https://mlsanity.com/paper/2407.02906/) (2024)
- [WorldGen: A Large Scale Generative Simulator](https://mlsanity.com/paper/2210.00715/) (2022)
- [Single-Scanline Relative Pose Estimation for Rolling Shutter Cameras](https://mlsanity.com/paper/2506.22069/) (2025)
- [Minimal Solvers for Full DoF Motion Estimation from Asynchronous Tracks](https://mlsanity.com/paper/2508.17537/) (2025)
- [Moment-Reenacting: Inverse Motion Degradation with Cross-shutter Guidance](https://mlsanity.com/paper/2605.22423/) (2026)
- [Let's Roll: Synthetic Dataset Analysis for Pedestrian Detection Across Different Shutter Types](https://mlsanity.com/paper/2309.08136/) (2023)
- [SpaceDG: Benchmarking Spatial Intelligence under Visual Degradation](https://mlsanity.com/paper/2605.22536/) (2026)

## Abstract (arXiv)

This paper proposes the first real-world rolling shutter (RS) correction dataset, BS-RSC, and a corresponding model to correct the RS frames in a distorted video. Mobile devices in the consumer market with CMOS-based sensors for video capture often result in rolling shutter effects when relative movements occur during the video acquisition process, calling for RS effect removal techniques. However, current state-of-the-art RS correction methods often fail to remove RS effects in real scenarios since the motions are various and hard to model. To address this issue, we propose a real-world RS correction dataset BS-RSC. Real distorted videos with corresponding ground truth are recorded simultaneously via a well-designed beam-splitter-based acquisition system. BS-RSC contains various motions of both camera and objects in dynamic scenes. Further, an RS correction model with adaptive warping is proposed. Our model can warp the learned RS features into global shutter counterparts adaptively with predicted multiple displacement fields. These warped features are aggregated and then reconstructed into high-quality global shutter frames in a coarse-to-fine strategy. Experimental results demonstrate the effectiveness of the proposed method, and our dataset can improve the model's ability to remove the RS effects in the real world.