# Bringing Old Films Back to Life

arXiv: https://arxiv.org/abs/2203.17276 · HTML: https://mlsanity.com/paper/2203.17276/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 55 references that are in the map.
- First posted on arXiv 31 Mar 2022.

## Builds on (most cited first, 30 of 41)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data](https://mlsanity.com/paper/2107.10833/) (2021)
- [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018)
- [Designing a Practical Degradation Model for Deep Blind Image Super-Resolution](https://mlsanity.com/paper/2103.14006/) (2021)
- [CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows](https://mlsanity.com/paper/2107.00652/) (2021)
- [FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising](https://mlsanity.com/paper/1710.04026/) (2017)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond](https://mlsanity.com/paper/2012.02181/) (2020)
- [Learning Blind Video Temporal Consistency](https://mlsanity.com/paper/1808.00449/) (2018)
- [BEVT: BERT Pretraining of Video Transformers](https://mlsanity.com/paper/2112.01529/) (2021)
- [Recurrent Back-Projection Network for Video Super-Resolution](https://mlsanity.com/paper/1903.10128/) (2019)
- [High-Fidelity Pluralistic Image Completion with Transformers](https://mlsanity.com/paper/2103.14031/) (2021)
- [FastDVDnet: Towards Real-Time Deep Video Denoising Without Flow Estimation](https://mlsanity.com/paper/1907.01361/) (2019)
- [Deep Flow-Guided Video Inpainting](https://mlsanity.com/paper/1905.02884/) (2019)
- [Deep Video Inpainting](https://mlsanity.com/paper/1905.01639/) (2019)
- [Spatio-Temporal Filter Adaptive Network for Video Deblurring](https://mlsanity.com/paper/1904.12257/) (2019)
- [Flow-edge Guided Video Completion](https://mlsanity.com/paper/2009.01835/) (2020)
- [Coherent Online Video Style Transfer](https://mlsanity.com/paper/1703.09211/) (2017)

## Built on by (most cited first, 30 of 31)

- [Recurrent Video Restoration Transformer with Guided Deformable Attention](https://mlsanity.com/paper/2206.02146/) (2022)
- [Blind Video Deflickering by Neural Filtering with a Flawed Atlas](https://mlsanity.com/paper/2303.08120/) (2023)
- [CAD: Photorealistic 3D Generation via Adversarial Distillation](https://mlsanity.com/paper/2312.06663/) (2023)
- [BiSTNet: Semantic Image Prior Guided Bidirectional Temporal Feature Fusion for Deep Exemplar-based Video Colorization](https://mlsanity.com/paper/2212.02268/) (2022)
- [Transformer-based Generative Adversarial Networks in Computer Vision: A Comprehensive Survey](https://mlsanity.com/paper/2302.08641/) (2023)
- [ColorMNet: A Memory-based Deep Spatial-Temporal Feature Propagation Network for Video Colorization](https://mlsanity.com/paper/2404.06251/) (2024)
- [MultiColor: Image Colorization by Learning from Multiple Color Spaces](https://mlsanity.com/paper/2408.04172/) (2024)
- [ChromaDistill: Colorizing Monochrome Radiance Fields with Knowledge Distillation](https://mlsanity.com/paper/2309.07668/) (2023)
- [Modernizing Old Photos Using Multiple References via Photorealistic Style Transfer](https://mlsanity.com/paper/2304.04461/) (2023)
- [Simulating analogue film damage to analyse and improve artefact restoration on high-resolution scans](https://mlsanity.com/paper/2302.10004/) (2023)
- [SVCNet: Scribble-based Video Colorization Network with Temporal Aggregation](https://mlsanity.com/paper/2303.11591/) (2023)
- [Reference-based Restoration of Digitized Analog Videotapes](https://mlsanity.com/paper/2310.14926/) (2023)
- [Semantic Lens: Instance-Centric Semantic Alignment for Video Super-Resolution](https://mlsanity.com/paper/2312.07823/) (2023)
- [ReBotNet: Fast Real-time Video Enhancement](https://mlsanity.com/paper/2303.13504/) (2023)
- [Restoration of Analog Videos Using Swin-UNet](https://mlsanity.com/paper/2311.04261/) (2023)
- [HyperColorization: Propagating spatially sparse noisy spectral clues for reconstructing hyperspectral images](https://mlsanity.com/paper/2403.11935/) (2024)
- [L-C4: Language-Based Video Colorization for Creative and Consistent Color](https://mlsanity.com/paper/2410.04972/) (2024)
- [Coloring the Noise: Adversarial Sobolev Alignment for Faithful Image Super Resolution](https://mlsanity.com/paper/2605.23264/) (2026)
- [AbsoluteDegradation: A Physics-Inspired Synthetic Film-Degradation Pipeline and Archival Film Restoration Benchmark](https://mlsanity.com/paper/2607.02131/) (2026)
- [Loop-Mamba: A Loop Mamba with Degradation-Aware and Shared Memory for Old Photo Restoration](https://mlsanity.com/paper/2608.02346/) (2026)
- [BlazeBVD: Make Scale-Time Equalization Great Again for Blind Video Deflickering](https://mlsanity.com/paper/2403.06243/) (2024)
- [LatentColorization: Latent Diffusion-Based Speaker Video Colorization](https://mlsanity.com/paper/2405.05707/) (2024)
- [ControlCol: Controllability in Automatic Speaker Video Colorization](https://mlsanity.com/paper/2408.11711/) (2024)
- [Adaptive Domain Learning for Cross-domain Image Denoising](https://mlsanity.com/paper/2411.01472/) (2024)
- [MambaVSR: Content-Aware Scanning State Space Model for Video Super-Resolution](https://mlsanity.com/paper/2506.11768/) (2025)
- [PixelBoost: Leveraging Brownian Motion for Realistic-Image Super-Resolution](https://mlsanity.com/paper/2506.23254/) (2025)
- [Instant Video Models: Universal Adapters for Stabilizing Image-Based Networks](https://mlsanity.com/paper/2512.03014/) (2025)
- [HaineiFRDM: Structure-Preserving Diffusion for Film Restoration under Fast Motion and Diverse Defects](https://mlsanity.com/paper/2512.24946/) (2025)
- [Improving Image-to-Image Translation via a Rectified Flow Reformulation](https://mlsanity.com/paper/2603.20186/) (2026)
- [SW-A2-Bench: Benchmarking Autonomous Software Agent Generation for Agentic Web](https://mlsanity.com/paper/2604.04226/) (2026)

## Abstract (arXiv)

We present a learning-based framework, recurrent transformer network (RTN), to restore heavily degraded old films. Instead of performing frame-wise restoration, our method is based on the hidden knowledge learned from adjacent frames that contain abundant information about the occlusion, which is beneficial to restore challenging artifacts of each frame while ensuring temporal coherency. Moreover, contrasting the representation of the current frame and the hidden knowledge makes it possible to infer the scratch position in an unsupervised manner, and such defect localization generalizes well to real-world degradations. To better resolve mixed degradation and compensate for the flow estimation error during frame alignment, we propose to leverage more expressive transformer blocks for spatial restoration. Experiments on both synthetic dataset and real-world old films demonstrate the significant superiority of the proposed RTN over existing solutions. In addition, the same framework can effectively propagate the color from keyframes to the whole video, ultimately yielding compelling restored films. The implementation and model will be released at https://github.com/raywzy/Bringing-Old-Films-Back-to-Life.