# Towards Nonlinear-Motion-Aware and Occlusion-Robust Rolling Shutter Correction

arXiv: https://arxiv.org/abs/2303.18125 · HTML: https://mlsanity.com/paper/2303.18125/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 38 references that are in the map.
- First posted on arXiv 31 Mar 2023.

## Builds on (most cited first, 18 of 18)

- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [Softmax Splatting for Video Frame Interpolation](https://mlsanity.com/paper/2003.05534/) (2020)
- [Learning to Estimate Hidden Motions with Global Motion Aggregation](https://mlsanity.com/paper/2104.02409/) (2021)
- [AdaCoF: Adaptive Collaboration of Flows for Video Frame Interpolation](https://mlsanity.com/paper/1907.10244/) (2019)
- [Video Frame Interpolation Transformer](https://mlsanity.com/paper/2111.13817/) (2021)
- [Rolling-Shutter-Aware Differential SfM and Image Rectification](https://mlsanity.com/paper/1903.03943/) (2019)
- [Towards Rolling Shutter Correction and Deblurring in Dynamic Scenes](https://mlsanity.com/paper/2104.01601/) (2021)
- [Rolling Shutter Camera Relative Pose: Generalized Epipolar Geometry](https://mlsanity.com/paper/1605.00475/) (2016)
- [SUNet: Symmetric Undistortion Network for Rolling Shutter Correction](https://mlsanity.com/paper/2108.04775/) (2021)
- [Context-Aware Video Reconstruction for Rolling Shutter Cameras](https://mlsanity.com/paper/2205.12912/) (2022)
- [Learning Adaptive Warping for Real-World Rolling Shutter Correction](https://mlsanity.com/paper/2204.13886/) (2022)
- [Revisiting Rolling Shutter Bundle Adjustment: Toward Accurate and Fast Solution](https://mlsanity.com/paper/2209.08503/) (2022)
- [Minimal Solvers for Monocular Rolling Shutter Compensation under Ackermann Motion](https://mlsanity.com/paper/1712.03159/) (2017)
- [Image Stitching and Rectification for Hand-Held Cameras](https://mlsanity.com/paper/2008.09229/) (2020)
- [Neural Global Shutter: Learn to Restore Video from a Rolling Shutter Camera with Global Reset Feature](https://mlsanity.com/paper/2204.00974/) (2022)
- [Combining Internal and External Constraints for Unrolling Shutter in Videos](https://mlsanity.com/paper/2207.11725/) (2022)

## Built on by (most cited first, 8 of 8)

- [Learning Occlusion-Robust Vision Transformers for Real-Time UAV Tracking](https://mlsanity.com/paper/2504.09228/) (2025)
- [Implicit Event-RGBD Neural SLAM](https://mlsanity.com/paper/2311.11013/) (2023)
- [HiMo: High-Speed Objects Motion Compensation in Point Clouds](https://mlsanity.com/paper/2503.00803/) (2025)
- [Single-Scanline Relative Pose Estimation for Rolling Shutter Cameras](https://mlsanity.com/paper/2506.22069/) (2025)
- [Minimal Solvers for Full DoF Motion Estimation from Asynchronous Tracks](https://mlsanity.com/paper/2508.17537/) (2025)
- [Rolling Shutter Camera Self-Calibration](https://mlsanity.com/paper/2608.01509/) (2026)
- [Scanline-Aware Animatable Gaussian Avatars from Rolling-Shutter Videos](https://mlsanity.com/paper/2608.17314/) (2026)
- [HaRP: High Dynamic Range Photosequencing through Dual Reversed Shutter Scanning](https://mlsanity.com/paper/2609.28439/) (2026)

## Abstract (arXiv)

This paper addresses the problem of rolling shutter correction in complex nonlinear and dynamic scenes with extreme occlusion. Existing methods suffer from two main drawbacks. Firstly, they face challenges in estimating the accurate correction field due to the uniform velocity assumption, leading to significant image correction errors under complex motion. Secondly, the drastic occlusion in dynamic scenes prevents current solutions from achieving better image quality because of the inherent difficulties in aligning and aggregating multiple frames. To tackle these challenges, we model the curvilinear trajectory of pixels analytically and propose a geometry-based Quadratic Rolling Shutter (QRS) motion solver, which precisely estimates the high-order correction field of individual pixels. Besides, to reconstruct high-quality occlusion frames in dynamic scenes, we present a 3D video architecture that effectively Aligns and Aggregates multi-frame context, namely, RSA2-Net. We evaluate our method across a broad range of cameras and video sequences, demonstrating its significant superiority. Specifically, our method surpasses the state-of-the-art by +4.98, +0.77, and +4.33 of PSNR on Carla-RS, Fastec-RS, and BS-RSC datasets, respectively. Code is available at https://github.com/DelinQu/qrsc.