# Stacked Deep Multi-Scale Hierarchical Network for Fast Bokeh Effect Rendering from a Single Image

arXiv: https://arxiv.org/abs/2105.07174 · HTML: https://mlsanity.com/paper/2105.07174/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 26 references that are in the map.
- First posted on arXiv 15 May 2021.

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [Selective Kernel Networks](https://mlsanity.com/paper/1903.06586/) (2019)
- [Deep Stacked Hierarchical Multi-patch Network for Image Deblurring](https://mlsanity.com/paper/1904.03468/) (2019)
- [AI Benchmark: Running Deep Neural Networks on Android Smartphones](https://mlsanity.com/paper/1810.01109/) (2018)
- [Synthetic Depth-of-Field with a Single-Camera Mobile Phone](https://mlsanity.com/paper/1806.04171/) (2018)
- [Residual Conv-Deconv Grid Network for Semantic Segmentation](https://mlsanity.com/paper/1707.07958/) (2017)
- [Rendering Natural Camera Bokeh Effect with Deep Learning](https://mlsanity.com/paper/2006.05698/) (2020)
- [NTIRE 2020 Challenge on NonHomogeneous Dehazing](https://mlsanity.com/paper/2005.03457/) (2020)
- [SteReFo: Efficient Image Refocusing with Stereo Vision](https://mlsanity.com/paper/1909.13395/) (2019)
- [Fast Deep Multi-patch Hierarchical Network for Nonhomogeneous Image Dehazing](https://mlsanity.com/paper/2005.05999/) (2020)
- [DeepLens: Shallow Depth Of Field From A Single Image](https://mlsanity.com/paper/1810.08100/) (2018)
- [Depth-aware Blending of Smoothed Images for Bokeh Effect Generation](https://mlsanity.com/paper/2005.14214/) (2020)

## Built on by (most cited first, 18 of 18)

- [BokehMe: When Neural Rendering Meets Classical Rendering](https://mlsanity.com/paper/2206.12614/) (2022)
- [DoF-NeRF: Depth-of-Field Meets Neural Radiance Fields](https://mlsanity.com/paper/2208.00945/) (2022)
- [MPIB: An MPI-Based Bokeh Rendering Framework for Realistic Partial Occlusion Effects](https://mlsanity.com/paper/2207.08403/) (2022)
- [Bokehlicious: Photorealistic Bokeh Rendering with Controllable Apertures](https://mlsanity.com/paper/2503.16067/) (2025)
- [BokehDiff: Neural Lens Blur with One-Step Diffusion](https://mlsanity.com/paper/2507.18060/) (2025)
- [DoF-Gaussian: Controllable Depth-of-Field for 3D Gaussian Splatting](https://mlsanity.com/paper/2503.00746/) (2025)
- [DiffCamera: Arbitrary Refocusing on Images](https://mlsanity.com/paper/2509.26599/) (2025)
- [Natural & Adversarial Bokeh Rendering via Circle-of-Confusion Predictive Network](https://mlsanity.com/paper/2111.12971/) (2021)
- [Bokeh-Loss GAN: Multi-Stage Adversarial Training for Realistic Edge-Aware Bokeh](https://mlsanity.com/paper/2208.12343/) (2022)
- [Adaptive Mask-based Pyramid Network for Realistic Bokeh Rendering](https://mlsanity.com/paper/2210.16078/) (2022)
- [BokehOrNot: Transforming Bokeh Effect with Image Transformer and Lens Metadata Embedding](https://mlsanity.com/paper/2306.04032/) (2023)
- [Realistic Bokeh Effect Rendering on Mobile GPUs, Mobile AI & AIM 2022 challenge: Report](https://mlsanity.com/paper/2211.06769/) (2022)
- [The First Controllable Bokeh Rendering Challenge at NTIRE 2026](https://mlsanity.com/paper/2605.05510/) (2026)
- [Variable Aperture Bokeh Rendering via Customized Focal Plane Guidance](https://mlsanity.com/paper/2410.14400/) (2024)
- [Any-to-Bokeh: Arbitrary-Subject Video Refocusing with Video Diffusion Model](https://mlsanity.com/paper/2505.21593/) (2025)
- [Direct Handheld Burst Imaging to Simulated Defocus](https://mlsanity.com/paper/2207.04175/) (2022)
- [Bokeh Rendering Based on Adaptive Depth Calibration Network](https://mlsanity.com/paper/2302.10808/) (2023)
- [GBSD: Generative Bokeh with Stage Diffusion](https://mlsanity.com/paper/2306.08251/) (2023)

## Abstract (arXiv)

The Bokeh Effect is one of the most desirable effects in photography for rendering artistic and aesthetic photos. Usually, it requires a DSLR camera with different aperture and shutter settings and certain photography skills to generate this effect. In smartphones, computational methods and additional sensors are used to overcome the physical lens and sensor limitations to achieve such effect. Most of the existing methods utilized additional sensor's data or pretrained network for fine depth estimation of the scene and sometimes use portrait segmentation pretrained network module to segment salient objects in the image. Because of these reasons, networks have many parameters, become runtime intensive and unable to run in mid-range devices. In this paper, we used an end-to-end Deep Multi-Scale Hierarchical Network (DMSHN) model for direct Bokeh effect rendering of images captured from the monocular camera. To further improve the perceptual quality of such effect, a stacked model consisting of two DMSHN modules is also proposed. Our model does not rely on any pretrained network module for Monocular Depth Estimation or Saliency Detection, thus significantly reducing the size of model and run time. Stacked DMSHN achieves state-of-the-art results on a large scale EBB! dataset with around 6x less runtime compared to the current state-of-the-art model in processing HD quality images.