# Bringing Rolling Shutter Images Alive with Dual Reversed Distortion

arXiv: https://arxiv.org/abs/2203.06451 · HTML: https://mlsanity.com/paper/2203.06451/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 40 references that are in the map.
- First posted on arXiv 12 Mar 2022.

## Builds on (most cited first, 20 of 20)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring](https://mlsanity.com/paper/1612.02177/) (2016)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [Softmax Splatting for Video Frame Interpolation](https://mlsanity.com/paper/2003.05534/) (2020)
- [Depth-Aware Video Frame Interpolation](https://mlsanity.com/paper/1904.00830/) (2019)
- [Bringing a Blurry Frame Alive at High Frame-Rate with an Event Camera](https://mlsanity.com/paper/1811.10180/) (2018)
- [FLAVR: Flow-Agnostic Video Representations for Fast Frame Interpolation](https://mlsanity.com/paper/2012.08512/) (2020)
- [Learning to Extract a Video Sequence from a Single Motion-Blurred Image](https://mlsanity.com/paper/1804.04065/) (2018)
- [Blurry Video Frame Interpolation](https://mlsanity.com/paper/2002.12259/) (2020)
- [Bringing Alive Blurred Moments](https://mlsanity.com/paper/1804.02913/) (2018)
- [Rolling-Shutter-Aware Differential SfM and Image Rectification](https://mlsanity.com/paper/1903.03943/) (2019)
- [Rolling Shutter Camera Relative Pose: Generalized Epipolar Geometry](https://mlsanity.com/paper/1605.00475/) (2016)
- [From two rolling shutters to one global shutter](https://mlsanity.com/paper/2006.01964/) (2020)
- [SUNet: Symmetric Undistortion Network for Rolling Shutter Correction](https://mlsanity.com/paper/2108.04775/) (2021)

## Built on by (most cited first, 15 of 15)

- [Blur Interpolation Transformer for Real-World Motion from Blur](https://mlsanity.com/paper/2211.11423/) (2022)
- [Deep Learning for Camera Calibration and Beyond: A Survey](https://mlsanity.com/paper/2303.10559/) (2023)
- [Context-Aware Video Reconstruction for Rolling Shutter Cameras](https://mlsanity.com/paper/2205.12912/) (2022)
- [USB-NeRF: Unrolling Shutter Bundle Adjusted Neural Radiance Fields](https://mlsanity.com/paper/2310.02687/) (2023)
- [RS-NeRF: Neural Radiance Fields from Rolling Shutter Images](https://mlsanity.com/paper/2407.10267/) (2024)
- [Self-supervised Learning to Bring Dual Reversed Rolling Shutter Images Alive](https://mlsanity.com/paper/2305.19862/) (2023)
- [Neural Global Shutter: Learn to Restore Video from a Rolling Shutter Camera with Global Reset Feature](https://mlsanity.com/paper/2204.00974/) (2022)
- [Single Image Rolling Shutter Removal with Diffusion Models](https://mlsanity.com/paper/2407.02906/) (2024)
- [Motion Blur Decomposition with Cross-shutter Guidance](https://mlsanity.com/paper/2404.01120/) (2024)
- [Self-Supervised Scene Dynamic Recovery from Rolling Shutter Images and Events](https://mlsanity.com/paper/2304.06930/) (2023)
- [Moment-Reenacting: Inverse Motion Degradation with Cross-shutter Guidance](https://mlsanity.com/paper/2605.22423/) (2026)
- [Velocity Disambiguation for Video Frame Interpolation](https://mlsanity.com/paper/2311.08007/) (2023)
- [SelfDRSC++: Self-Supervised Dual Reversed Rolling Shutter Correction via Video Interpolation](https://mlsanity.com/paper/2408.11411/) (2024)
- [Scanline-Aware Animatable Gaussian Avatars from Rolling-Shutter Videos](https://mlsanity.com/paper/2608.17314/) (2026)
- [HaRP: High Dynamic Range Photosequencing through Dual Reversed Shutter Scanning](https://mlsanity.com/paper/2609.28439/) (2026)

## Abstract (arXiv)

Rolling shutter (RS) distortion can be interpreted as the result of picking a row of pixels from instant global shutter (GS) frames over time during the exposure of the RS camera. This means that the information of each instant GS frame is partially, yet sequentially, embedded into the row-dependent distortion. Inspired by this fact, we address the challenging task of reversing this process, i.e., extracting undistorted GS frames from images suffering from RS distortion. However, since RS distortion is coupled with other factors such as readout settings and the relative velocity of scene elements to the camera, models that only exploit the geometric correlation between temporally adjacent images suffer from poor generality in processing data with different readout settings and dynamic scenes with both camera motion and object motion. In this paper, instead of two consecutive frames, we propose to exploit a pair of images captured by dual RS cameras with reversed RS directions for this highly challenging task. Grounded on the symmetric and complementary nature of dual reversed distortion, we develop a novel end-to-end model, IFED, to generate dual optical flow sequence through iterative learning of the velocity field during the RS time. Extensive experimental results demonstrate that IFED is superior to naive cascade schemes, as well as the state-of-the-art which utilizes adjacent RS images. Most importantly, although it is trained on a synthetic dataset, IFED is shown to be effective at retrieving GS frame sequences from real-world RS distorted images of dynamic scenes. Code is available at https://github.com/zzh-tech/Dual-Reversed-RS.