# Learning to Deblur using Light Field Generated and Real Defocus Images

arXiv: https://arxiv.org/abs/2204.00367 · HTML: https://mlsanity.com/paper/2204.00367/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 46 references that are in the map.
- First posted on arXiv 1 Apr 2022.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [Dynamic Filter Networks](https://mlsanity.com/paper/1605.09673/) (2016)
- [Mask TextSpotter: An End-to-End Trainable Neural Network for Spotting Text with Arbitrary Shapes](https://mlsanity.com/paper/1908.08207/) (2019)
- [Burst Denoising with Kernel Prediction Networks](https://mlsanity.com/paper/1712.02327/) (2017)
- [Defocus Deblurring Using Dual-Pixel Data](https://mlsanity.com/paper/2005.00305/) (2020)
- [Spatio-Temporal Filter Adaptive Network for Video Deblurring](https://mlsanity.com/paper/1904.12257/) (2019)
- [Iterative Filter Adaptive Network for Single Image Defocus Deblurring](https://mlsanity.com/paper/2108.13610/) (2021)
- [Unified Dynamic Convolutional Network for Super-Resolution with Variational Degradations](https://mlsanity.com/paper/2004.06965/) (2020)
- [Single Image Defocus Deblurring Using Kernel-Sharing Parallel Atrous Convolutions](https://mlsanity.com/paper/2108.09108/) (2021)
- [Learning to Reduce Defocus Blur by Realistically Modeling Dual-Pixel Data](https://mlsanity.com/paper/2012.03255/) (2020)
- [KOALAnet: Blind Super-Resolution using Kernel-Oriented Adaptive Local Adjustment](https://mlsanity.com/paper/2012.08103/) (2020)
- [Dual Pixel Exploration: Simultaneous Depth Estimation and Image Restoration](https://mlsanity.com/paper/2012.00301/) (2020)
- [A Unified Approach of Multi-scale Deep and Hand-crafted Features for Defocus Estimation](https://mlsanity.com/paper/1704.08992/) (2017)
- [Improving Single-Image Defocus Deblurring: How Dual-Pixel Images Help Through Multi-Task Learning](https://mlsanity.com/paper/2108.05251/) (2021)
- [Attention! Stay Focus!](https://mlsanity.com/paper/2104.07925/) (2021)

## Built on by (most cited first, 30 of 46)

- [Deblurring 3D Gaussian Splatting](https://mlsanity.com/paper/2401.00834/) (2024)
- [An Intelligent Agentic System for Complex Image Restoration Problems](https://mlsanity.com/paper/2410.17809/) (2024)
- [Enhancing Descriptive Image Quality Assessment with A Large-scale Multi-modal Dataset](https://mlsanity.com/paper/2405.18842/) (2024)
- [Visual-Instructed Degradation Diffusion for All-in-One Image Restoration](https://mlsanity.com/paper/2506.16960/) (2025)
- [Multi-Agent Image Restoration](https://mlsanity.com/paper/2503.09403/) (2025)
- [Multi-scale frequency separation network for image deblurring](https://mlsanity.com/paper/2206.00798/) (2022)
- [A Unified Framework for Microscopy Defocus Deblur with Multi-Pyramid Transformer and Contrastive Learning](https://mlsanity.com/paper/2403.02611/) (2024)
- [DC2: Dual-Camera Defocus Control by Learning to Refocus](https://mlsanity.com/paper/2304.03285/) (2023)
- [Sharp-NeRF: Grid-based Fast Deblurring Neural Radiance Fields Using Sharpness Prior](https://mlsanity.com/paper/2401.00825/) (2024)
- [Bokehlicious: Photorealistic Bokeh Rendering with Controllable Apertures](https://mlsanity.com/paper/2503.16067/) (2025)
- [Image Deblurring by Exploring In-depth Properties of Transformer](https://mlsanity.com/paper/2303.15198/) (2023)
- [Reversing Flow for Image Restoration](https://mlsanity.com/paper/2506.16961/) (2025)
- [Revisiting Image Deblurring with an Efficient ConvNet](https://mlsanity.com/paper/2302.02234/) (2023)
- [NTIRE 2024 Restore Any Image Model (RAIM) in the Wild Challenge](https://mlsanity.com/paper/2405.09923/) (2024)
- [Learning Single Image Defocus Deblurring with Misaligned Training Pairs](https://mlsanity.com/paper/2211.14502/) (2022)
- [Dynamic Neural Radiance Field From Defocused Monocular Video](https://mlsanity.com/paper/2407.05586/) (2024)
- [Restore-R1: Efficient Image Restoration Agents via Reinforcement Learning with Multimodal LLM Perceptual Feedback](https://mlsanity.com/paper/2512.18599/) (2025)
- [Bridge Frame and Event: Common Spatiotemporal Fusion for High-Dynamic Scene Optical Flow](https://mlsanity.com/paper/2503.06992/) (2025)
- [Efficient Depth- and Spatially-Varying Image Simulation for Defocus Deblur](https://mlsanity.com/paper/2507.00372/) (2025)
- [Generative Refocusing: Flexible Defocus Control from a Single Image](https://mlsanity.com/paper/2512.16923/) (2025)
- [TPGDiff: Hierarchical Triple-Prior Guided Diffusion for Image Restoration](https://mlsanity.com/paper/2601.20306/) (2026)
- [Self-Evolving Agentic Image Restoration via Deliberate Planning and Intuitive Execution](https://mlsanity.com/paper/2606.28971/) (2026)
- [Efficient Image Deblurring Networks based on Diffusion Models](https://mlsanity.com/paper/2401.05907/) (2024)
- [Fast and Accurate Image Restoration and Generation with Rank Enhanced Linear Attention](https://mlsanity.com/paper/2505.16157/) (2025)
- [PaAgent: Portrait-Aware Image Restoration Agent via Subjective-Objective Reinforcement Learning](https://mlsanity.com/paper/2603.17055/) (2026)
- [ConStyle v2: A Strong Prompter for All-in-One Image Restoration](https://mlsanity.com/paper/2406.18242/) (2024)
- [Continual Test-Time Adaptation for Single Image Defocus Deblurring via Causal Siamese Networks](https://mlsanity.com/paper/2501.09052/) (2025)
- [OAFuser: Towards Omni-Aperture Fusion for Light Field Semantic Segmentation](https://mlsanity.com/paper/2307.15588/) (2023)
- [DaBiT: Depth and Blur informed Transformer for Video Focal Deblurring](https://mlsanity.com/paper/2407.01230/) (2024)
- [A Sharpness Based Loss Function for Removing Out-of-Focus Blur](https://mlsanity.com/paper/2408.06014/) (2024)

## Abstract (arXiv)

Defocus deblurring is a challenging task due to the spatially varying nature of defocus blur. While deep learning approach shows great promise in solving image restoration problems, defocus deblurring demands accurate training data that consists of all-in-focus and defocus image pairs, which is difficult to collect. Naive two-shot capturing cannot achieve pixel-wise correspondence between the defocused and all-in-focus image pairs. Synthetic aperture of light fields is suggested to be a more reliable way to generate accurate image pairs. However, the defocus blur generated from light field data is different from that of the images captured with a traditional digital camera. In this paper, we propose a novel deep defocus deblurring network that leverages the strength and overcomes the shortcoming of light fields. We first train the network on a light field-generated dataset for its highly accurate image correspondence. Then, we fine-tune the network using feature loss on another dataset collected by the two-shot method to alleviate the differences between the defocus blur exists in the two domains. This strategy is proved to be highly effective and able to achieve the state-of-the-art performance both quantitatively and qualitatively on multiple test sets. Extensive ablation studies have been conducted to analyze the effect of each network module to the final performance.