# Defocus to focus: Photo-realistic bokeh rendering by fusing defocus and radiance priors

arXiv: https://arxiv.org/abs/2306.04506 · HTML: https://mlsanity.com/paper/2306.04506/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 69 references that are in the map.
- First posted on arXiv 7 Jun 2023.

## Builds on (most cited first, 18 of 18)

- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Estimating Depth from Monocular Images as Classification Using Deep Fully Convolutional Residual Networks](https://mlsanity.com/paper/1605.02305/) (2016)
- [Synthetic Depth-of-Field with a Single-Camera Mobile Phone](https://mlsanity.com/paper/1806.04171/) (2018)
- [GP-GAN: Towards Realistic High-Resolution Image Blending](https://mlsanity.com/paper/1703.07195/) (2017)
- [Deep Image Blending](https://mlsanity.com/paper/1910.11495/) (2019)
- [Rendering Natural Camera Bokeh Effect with Deep Learning](https://mlsanity.com/paper/2006.05698/) (2020)
- [Synthetic Defocus and Look-Ahead Autofocus for Casual Videography](https://mlsanity.com/paper/1905.06326/) (2019)
- [Learning to Autofocus](https://mlsanity.com/paper/2004.12260/) (2020)
- [AIM 2020 Challenge on Rendering Realistic Bokeh](https://mlsanity.com/paper/2011.04988/) (2020)
- [A Unified Approach of Multi-scale Deep and Hand-crafted Features for Defocus Estimation](https://mlsanity.com/paper/1704.08992/) (2017)
- [SteReFo: Efficient Image Refocusing with Stereo Vision](https://mlsanity.com/paper/1909.13395/) (2019)
- [Aperture Supervision for Monocular Depth Estimation](https://mlsanity.com/paper/1711.07933/) (2017)
- [BGGAN: Bokeh-Glass Generative Adversarial Network for Rendering Realistic Bokeh](https://mlsanity.com/paper/2011.02242/) (2020)
- [Monocular Depth Estimation with Augmented Ordinal Depth Relationships](https://mlsanity.com/paper/1806.00585/) (2018)
- [DeepLens: Shallow Depth Of Field From A Single Image](https://mlsanity.com/paper/1810.08100/) (2018)
- [Depth-aware Blending of Smoothed Images for Bokeh Effect Generation](https://mlsanity.com/paper/2005.14214/) (2020)

## Built on by (most cited first, 8 of 8)

- [Bokehlicious: Photorealistic Bokeh Rendering with Controllable Apertures](https://mlsanity.com/paper/2503.16067/) (2025)
- [Dynamic Neural Radiance Field From Defocused Monocular Video](https://mlsanity.com/paper/2407.05586/) (2024)
- [BokehFlow: Depth-Free Controllable Bokeh Rendering via Flow Matching](https://mlsanity.com/paper/2511.15066/) (2025)
- [The First Controllable Bokeh Rendering Challenge at NTIRE 2026](https://mlsanity.com/paper/2605.05510/) (2026)
- [Boosting Monocular Metric Depth Estimation via Bokeh Rendering](https://mlsanity.com/paper/2512.12425/) (2025)
- [LensStyle: Learning the Optical Aesthetics for Controllable Stylized Lens Effect Rendering](https://mlsanity.com/paper/2609.04939/) (2026)
- [A Novel Defocus-Blur Region Detection Approach Based on DCT Feature and PCNN Structure](https://mlsanity.com/paper/2311.12845/) (2023)
- [Dual-Camera All-in-Focus Neural Radiance Fields](https://mlsanity.com/paper/2504.16636/) (2025)

## Abstract (arXiv)

We consider the problem of realistic bokeh rendering from a single all-in-focus image. Bokeh rendering mimics aesthetic shallow depth-of-field (DoF) in professional photography, but these visual effects generated by existing methods suffer from simple flat background blur and blurred in-focus regions, giving rise to unrealistic rendered results. In this work, we argue that realistic bokeh rendering should (i) model depth relations and distinguish in-focus regions, (ii) sustain sharp in-focus regions, and (iii) render physically accurate Circle of Confusion (CoC). To this end, we present a Defocus to Focus (D2F) framework to learn realistic bokeh rendering by fusing defocus priors with the all-in-focus image and by implementing radiance priors in layered fusion. Since no depth map is provided, we introduce defocus hallucination to integrate depth by learning to focus. The predicted defocus map implies the blur amount of bokeh and is used to guide weighted layered rendering. In layered rendering, we fuse images blurred by different kernels based on the defocus map. To increase the reality of the bokeh, we adopt radiance virtualization to simulate scene radiance. The scene radiance used in weighted layered rendering reassigns weights in the soft disk kernel to produce the CoC. To ensure the sharpness of in-focus regions, we propose to fuse upsampled bokeh images and original images. We predict the initial fusion mask from our defocus map and refine the mask with a deep network. We evaluate our model on a large-scale bokeh dataset. Extensive experiments show that our approach is capable of rendering visually pleasing bokeh effects in complex scenes. In particular, our solution receives the runner-up award in the AIM 2020 Rendering Realistic Bokeh Challenge.