# Zoom To Learn, Learn To Zoom

arXiv: https://arxiv.org/abs/1905.05169 · HTML: https://mlsanity.com/paper/1905.05169/

## Key facts

- Cited by 115 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 May 2019.

## Built on by (most cited first, 30 of 115)

- [V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs](https://mlsanity.com/paper/2312.14135/) (2023)
- [Depth Pro: Sharp Monocular Metric Depth in Less Than a Second](https://mlsanity.com/paper/2410.02073/) (2024)
- [Efficient and Degradation-Adaptive Network for Real-World Image Super-Resolution](https://mlsanity.com/paper/2203.14216/) (2022)
- [Invertible Image Signal Processing](https://mlsanity.com/paper/2103.15061/) (2021)
- [Deep Burst Super-Resolution](https://mlsanity.com/paper/2101.10997/) (2021)
- [Multiscale Structure Guided Diffusion for Image Deblurring](https://mlsanity.com/paper/2212.01789/) (2022)
- [Real-World Single Image Super-Resolution: A Brief Review](https://mlsanity.com/paper/2103.02368/) (2021)
- [DreamClear: High-Capacity Real-World Image Restoration with Privacy-Safe Dataset Curation](https://mlsanity.com/paper/2410.18666/) (2024)
- [Learning to Upsample by Learning to Sample](https://mlsanity.com/paper/2308.15085/) (2023)
- [Realistic Blur Synthesis for Learning Image Deblurring](https://mlsanity.com/paper/2202.08771/) (2022)
- [Learning RAW-to-sRGB Mappings with Inaccurately Aligned Supervision](https://mlsanity.com/paper/2108.08119/) (2021)
- [Text Prior Guided Scene Text Image Super-resolution](https://mlsanity.com/paper/2106.15368/) (2021)
- [A Text Attention Network for Spatial Deformation Robust Scene Text Image Super-resolution](https://mlsanity.com/paper/2203.09388/) (2022)
- [Dual-Camera Super-Resolution with Aligned Attention Modules](https://mlsanity.com/paper/2109.01349/) (2021)
- [Towards Rolling Shutter Correction and Deblurring in Dynamic Scenes](https://mlsanity.com/paper/2104.01601/) (2021)
- [AligNeRF: High-Fidelity Neural Radiance Fields via Alignment-Aware Training](https://mlsanity.com/paper/2211.09682/) (2022)
- [Towards Efficient and Scale-Robust Ultra-High-Definition Image Demoireing](https://mlsanity.com/paper/2207.09935/) (2022)
- [Text Gestalt: Stroke-Aware Scene Text Image Super-Resolution](https://mlsanity.com/paper/2112.08171/) (2021)
- [SRWarp: Generalized Image Super-Resolution under Arbitrary Transformation](https://mlsanity.com/paper/2104.10325/) (2021)
- [Real-world Video Deblurring: A Benchmark Dataset and An Efficient Recurrent Neural Network](https://mlsanity.com/paper/2106.16028/) (2021)
- [Generative Photography: Scene-Consistent Camera Control for Realistic Text-to-Image Synthesis](https://mlsanity.com/paper/2412.02168/) (2024)
- [Variational Deep Image Restoration](https://mlsanity.com/paper/2207.01074/) (2022)
- [Learning Generative Structure Prior for Blind Text Image Super-resolution](https://mlsanity.com/paper/2303.14726/) (2023)
- [A Comprehensive Review of Deep Learning-based Single Image Super-resolution](https://mlsanity.com/paper/2102.09351/) (2021)
- [ParamISP: Learned Forward and Inverse ISPs using Camera Parameters](https://mlsanity.com/paper/2312.13313/) (2023)
- [Rethinking Image Editing Detection in the Era of Generative AI Revolution](https://mlsanity.com/paper/2311.17953/) (2023)
- [Self-Supervised Learning for Real-World Super-Resolution from Dual Zoomed Observations](https://mlsanity.com/paper/2203.01325/) (2022)
- [Reference-based Video Super-Resolution Using Multi-Camera Video Triplets](https://mlsanity.com/paper/2203.14537/) (2022)
- [Real-RawVSR: Real-World Raw Video Super-Resolution with a Benchmark Dataset](https://mlsanity.com/paper/2209.12475/) (2022)
- [Optical Aberration Correction in Postprocessing using Imaging Simulation](https://mlsanity.com/paper/2305.05867/) (2023)

## Abstract (arXiv)

This paper shows that when applying machine learning to digital zoom for photography, it is beneficial to use real, RAW sensor data for training. Existing learning-based super-resolution methods do not use real sensor data, instead operating on RGB images. In practice, these approaches result in loss of detail and accuracy in their digitally zoomed output when zooming in on distant image regions. We also show that synthesizing sensor data by resampling high-resolution RGB images is an oversimplified approximation of real sensor data and noise, resulting in worse image quality. The key barrier to using real sensor data for training is that ground truth high-resolution imagery is missing. We show how to obtain the ground-truth data with optically zoomed images and contribute a dataset, SR-RAW, for real-world computational zoom. We use SR-RAW to train a deep network with a novel contextual bilateral loss (CoBi) that delivers critical robustness to mild misalignment in input-output image pairs. The trained network achieves state-of-the-art performance in 4X and 8X computational zoom.