# Single Image Rolling Shutter Removal with Diffusion Models

arXiv: https://arxiv.org/abs/2407.02906 · HTML: https://mlsanity.com/paper/2407.02906/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 38 of its 76 references that are in the map.
- First posted on arXiv 3 Jul 2024.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 30 of 38)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022)
- [Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model](https://mlsanity.com/paper/2212.00490/) (2022)
- [Implicit Diffusion Models for Continuous Super-Resolution](https://mlsanity.com/paper/2303.16491/) (2023)
- [More Control for Free! Image Synthesis with Semantic Diffusion Guidance](https://mlsanity.com/paper/2112.05744/) (2021)
- [Conditional Image-to-Video Generation with Latent Flow Diffusion Models](https://mlsanity.com/paper/2303.13744/) (2023)
- [The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimation](https://mlsanity.com/paper/2306.01923/) (2023)
- [Deep Image Deblurring: A Survey](https://mlsanity.com/paper/2201.10700/) (2022)
- [MDM: Molecular Diffusion Model for 3D Molecule Generation](https://mlsanity.com/paper/2209.05710/) (2022)
- [Towards High-quality HDR Deghosting with Conditional Diffusion Models](https://mlsanity.com/paper/2311.00932/) (2023)
- [Rolling-Shutter-Aware Differential SfM and Image Rectification](https://mlsanity.com/paper/1903.03943/) (2019)
- [Towards Rolling Shutter Correction and Deblurring in Dynamic Scenes](https://mlsanity.com/paper/2104.01601/) (2021)
- [Rolling Shutter Camera Relative Pose: Generalized Epipolar Geometry](https://mlsanity.com/paper/1605.00475/) (2016)
- [RealFlow: EM-based Realistic Optical Flow Dataset Generation from Videos](https://mlsanity.com/paper/2207.11075/) (2022)
- [From two rolling shutters to one global shutter](https://mlsanity.com/paper/2006.01964/) (2020)
- [SUNet: Symmetric Undistortion Network for Rolling Shutter Correction](https://mlsanity.com/paper/2108.04775/) (2021)
- [Gyroscope-Aided Motion Deblurring with Deep Networks](https://mlsanity.com/paper/1810.00986/) (2018)
- [GyroFlow: Gyroscope-Guided Unsupervised Optical Flow Learning](https://mlsanity.com/paper/2103.13725/) (2021)
- [Bringing Rolling Shutter Images Alive with Dual Reversed Distortion](https://mlsanity.com/paper/2203.06451/) (2022)
- [Eliminating Contextual Prior Bias for Semantic Image Editing via Dual-Cycle Diffusion](https://mlsanity.com/paper/2302.02394/) (2023)

## Built on by (most cited first, 5 of 5)

- [LightenDiffusion: Unsupervised Low-Light Image Enhancement with Latent-Retinex Diffusion Models](https://mlsanity.com/paper/2407.08939/) (2024)
- [PointRegGPT: Boosting 3D Point Cloud Registration using Generative Point-Cloud Pairs for Training](https://mlsanity.com/paper/2407.14054/) (2024)
- [DMAligner: Enhancing Image Alignment via Diffusion Model Based View Synthesis](https://mlsanity.com/paper/2602.23022/) (2026)
- [StableMotion: One-Step Motion Estimation with Diffusion Prior](https://mlsanity.com/paper/2505.06668/) (2025)
- [ZeroIDIR: Zero-Reference Illumination Degradation Image Restoration with Perturbed Consistency Diffusion Models](https://mlsanity.com/paper/2605.11435/) (2026)

## Abstract (arXiv)

We present RS-Diffusion, the first Diffusion Models-based method for single-frame Rolling Shutter (RS) correction. RS artifacts compromise visual quality of frames due to the row-wise exposure of CMOS sensors. Most previous methods have focused on multi-frame approaches, using temporal information from consecutive frames for the motion rectification. However, few approaches address the more challenging but important single frame RS correction. In this work, we present an ``image-to-motion" framework via diffusion techniques, with a designed patch-attention module. In addition, we present the RS-Real dataset, comprised of captured RS frames alongside their corresponding Global Shutter (GS) ground-truth pairs. The GS frames are corrected from the RS ones, guided by the corresponding Inertial Measurement Unit (IMU) gyroscope data acquired during capture. Experiments show that RS-Diffusion surpasses previous single-frame RS methods, demonstrates the potential of diffusion-based approaches, and provides a valuable dataset for further research.