# Semantic Aware Diffusion Inverse Tone Mapping

arXiv: https://arxiv.org/abs/2405.15468 · HTML: https://mlsanity.com/paper/2405.15468/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 48 references that are in the map.
- First posted on arXiv 24 May 2024.

## Builds on (most cited first, 23 of 23)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [HDR image reconstruction from a single exposure using deep CNNs](https://mlsanity.com/paper/1710.07480/) (2017)
- [Single-Image HDR Reconstruction by Learning to Reverse the Camera Pipeline](https://mlsanity.com/paper/2004.01179/) (2020)
- [InverseRenderNet: Learning single image inverse rendering](https://mlsanity.com/paper/1811.12328/) (2018)
- [ExpandNet: A Deep Convolutional Neural Network for High Dynamic Range Expansion from Low Dynamic Range Content](https://mlsanity.com/paper/1803.02266/) (2018)
- [Image inpainting: A review](https://mlsanity.com/paper/1909.06399/) (2019)
- [Single Image HDR Reconstruction Using a CNN with Masked Features and Perceptual Loss](https://mlsanity.com/paper/2005.07335/) (2020)
- [FastFCN: Rethinking Dilated Convolution in the Backbone for Semantic Segmentation](https://mlsanity.com/paper/1903.11816/) (2019)
- [HDR-VDP-3: A multi-metric for predicting image differences, quality and contrast distortions in high dynamic range and regular content](https://mlsanity.com/paper/2304.13625/) (2023)
- [RawHDR: High Dynamic Range Image Reconstruction from a Single Raw Image](https://mlsanity.com/paper/2309.02020/) (2023)
- [Text2Light: Zero-Shot Text-Driven HDR Panorama Generation](https://mlsanity.com/paper/2209.09898/) (2022)
- [GlowGAN: Unsupervised Learning of HDR Images from LDR Images in the Wild](https://mlsanity.com/paper/2211.12352/) (2022)
- [Luminance Attentive Networks for HDR Image and Panorama Reconstruction](https://mlsanity.com/paper/2109.06688/) (2021)
- [Single Image LDR to HDR Conversion using Conditional Diffusion](https://mlsanity.com/paper/2307.02814/) (2023)
- [PyMatting: A Python Library for Alpha Matting](https://mlsanity.com/paper/2003.12382/) (2020)
- [G-SemTMO: Tone Mapping with a Trainable Semantic Graph](https://mlsanity.com/paper/2208.14113/) (2022)

## Built on by (most cited first, 7 of 7)

- [LEDiff: Latent Exposure Diffusion for HDR Generation](https://mlsanity.com/paper/2412.14456/) (2024)
- [UltraFusion: Ultra High Dynamic Imaging using Exposure Fusion](https://mlsanity.com/paper/2501.11515/) (2025)
- [AIM 2025 challenge on Inverse Tone Mapping Report: Methods and Results](https://mlsanity.com/paper/2508.13479/) (2025)
- [DiffHDR: Re-Exposing LDR Videos with Video Diffusion Models](https://mlsanity.com/paper/2604.06161/) (2026)
- [Boosting HDR Image Reconstruction via Semantic Knowledge Transfer](https://mlsanity.com/paper/2503.15361/) (2025)
- [A Cycle Ride to HDR: Semantics Aware Self-Supervised Framework for Unpaired LDR-to-HDR Image Reconstruction](https://mlsanity.com/paper/2410.15068/) (2024)
- [ExpandDiff: Dynamic Range Expanding Diffusion for Single-Image HDR Reconstruction](https://mlsanity.com/paper/2609.39624/) (2026)

## Abstract (arXiv)

The range of real-world scene luminance is larger than the capture capability of many digital camera sensors which leads to details being lost in captured images, most typically in bright regions. Inverse tone mapping attempts to boost these captured Standard Dynamic Range (SDR) images back to High Dynamic Range (HDR) by creating a mapping that linearizes the well exposed values from the SDR image, and provides a luminance boost to the clipped content. However, in most cases, the details in the clipped regions cannot be recovered or estimated. In this paper, we present a novel inverse tone mapping approach for mapping SDR images to HDR that generates lost details in clipped regions through a semantic-aware diffusion based inpainting approach. Our method proposes two major contributions - first, we propose to use a semantic graph to guide SDR diffusion based inpainting in masked regions in a saturated image. Second, drawing inspiration from traditional HDR imaging and bracketing methods, we propose a principled formulation to lift the SDR inpainted regions to HDR that is compatible with generative inpainting methods. Results show that our method demonstrates superior performance across different datasets on objective metrics, and subjective experiments show that the proposed method matches (and in most cases outperforms) state-of-art inverse tone mapping operators in terms of objective metrics and outperforms them for visual fidelity.