# BokehOrNot: Transforming Bokeh Effect with Image Transformer and Lens Metadata Embedding

arXiv: https://arxiv.org/abs/2306.04032 · HTML: https://mlsanity.com/paper/2306.04032/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 36 references that are in the map.
- First posted on arXiv 6 Jun 2023.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Segmenter: Transformer for Semantic Segmentation](https://mlsanity.com/paper/2105.05633/) (2021)
- [VRT: A Video Restoration Transformer](https://mlsanity.com/paper/2201.12288/) (2022)
- [Refusion: Enabling Large-Size Realistic Image Restoration with Latent-Space Diffusion Models](https://mlsanity.com/paper/2304.08291/) (2023)
- [Rendering Natural Camera Bokeh Effect with Deep Learning](https://mlsanity.com/paper/2006.05698/) (2020)
- [BokehMe: When Neural Rendering Meets Classical Rendering](https://mlsanity.com/paper/2206.12614/) (2022)
- [BSRT: Improving Burst Super-Resolution with Swin Transformer and Flow-Guided Deformable Alignment](https://mlsanity.com/paper/2204.08332/) (2022)
- [AIM 2020 Challenge on Rendering Realistic Bokeh](https://mlsanity.com/paper/2011.04988/) (2020)
- [BGGAN: Bokeh-Glass Generative Adversarial Network for Rendering Realistic Bokeh](https://mlsanity.com/paper/2011.02242/) (2020)
- [Stacked Deep Multi-Scale Hierarchical Network for Fast Bokeh Effect Rendering from a Single Image](https://mlsanity.com/paper/2105.07174/) (2021)
- [Perceptual Image Enhancement for Smartphone Real-Time Applications](https://mlsanity.com/paper/2210.13552/) (2022)
- [Adaptive Mask-based Pyramid Network for Realistic Bokeh Rendering](https://mlsanity.com/paper/2210.16078/) (2022)
- [Realistic Bokeh Effect Rendering on Mobile GPUs, Mobile AI & AIM 2022 challenge: Report](https://mlsanity.com/paper/2211.06769/) (2022)

## Built on by (most cited first, 3 of 3)

- [Bokehlicious: Photorealistic Bokeh Rendering with Controllable Apertures](https://mlsanity.com/paper/2503.16067/) (2025)
- [BokehDiff: Neural Lens Blur with One-Step Diffusion](https://mlsanity.com/paper/2507.18060/) (2025)
- [Variable Aperture Bokeh Rendering via Customized Focal Plane Guidance](https://mlsanity.com/paper/2410.14400/) (2024)

## Abstract (arXiv)

Bokeh effect is an optical phenomenon that offers a pleasant visual experience, typically generated by high-end cameras with wide aperture lenses. The task of bokeh effect transformation aims to produce a desired effect in one set of lenses and apertures based on another combination. Current models are limited in their ability to render a specific set of bokeh effects, primarily transformations from sharp to blur. In this paper, we propose a novel universal method for embedding lens metadata into the model and introducing a loss calculation method using alpha masks from the newly released Bokeh Effect Transformation Dataset(BETD) [3]. Based on the above techniques, we propose the BokehOrNot model, which is capable of producing both blur-to-sharp and sharp-to-blur bokeh effect with various combinations of lenses and aperture sizes. Our proposed model outperforms current leading bokeh rendering and image restoration models and renders visually natural bokeh effects. Our code is available at: https://github.com/indicator0/bokehornot.