# HDR Environment Map Estimation with Latent Diffusion Models

arXiv: https://arxiv.org/abs/2507.21261 · HTML: https://mlsanity.com/paper/2507.21261/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 46 of its 59 references that are in the map.
- First posted on arXiv 28 Jul 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 46)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Adversarial Diffusion Distillation](https://mlsanity.com/paper/2311.17042/) (2023)
- [SynSin: End-to-end View Synthesis from a Single Image](https://mlsanity.com/paper/1912.08804/) (2019)
- [Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling](https://mlsanity.com/paper/1908.00222/) (2019)
- [SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Model](https://mlsanity.com/paper/2212.05034/) (2022)
- [Large Scale Image Completion via Co-Modulated Generative Adversarial Networks](https://mlsanity.com/paper/2103.10428/) (2021)
- [Learning to Predict Indoor Illumination from a Single Image](https://mlsanity.com/paper/1704.00090/) (2017)
- [PanoFormer: Panorama Transformer for Indoor 360 Depth Estimation](https://mlsanity.com/paper/2203.09283/) (2022)
- [Deep Parametric Indoor Lighting Estimation](https://mlsanity.com/paper/1910.08812/) (2019)
- [Fast Spatially-Varying Indoor Lighting Estimation](https://mlsanity.com/paper/1906.03799/) (2019)
- [Deep Sky Modeling for Single Image Outdoor Lighting Estimation](https://mlsanity.com/paper/1905.03897/) (2019)
- [Lighthouse: Predicting Lighting Volumes for Spatially-Coherent Illumination](https://mlsanity.com/paper/2003.08367/) (2020)
- [StyleLight: HDR Panorama Generation for Lighting Estimation and Editing](https://mlsanity.com/paper/2207.14811/) (2022)
- [Neural Illumination: Lighting Prediction for Indoor Environments](https://mlsanity.com/paper/1906.07370/) (2019)
- [Kernel Transformer Networks for Compact Spherical Convolution](https://mlsanity.com/paper/1812.03115/) (2018)
- [Deep Outdoor Illumination Estimation](https://mlsanity.com/paper/1611.06403/) (2016)
- [PanoDiffusion: 360-degree Panorama Outpainting via Diffusion](https://mlsanity.com/paper/2307.03177/) (2023)
- [EMLight: Lighting Estimation via Spherical Distribution Approximation](https://mlsanity.com/paper/2012.11116/) (2020)
- [HDR Environment Map Estimation for Real-Time Augmented Reality](https://mlsanity.com/paper/2011.10687/) (2020)
- [Diverse Plausible 360-Degree Image Outpainting for Efficient 3DCG Background Creation](https://mlsanity.com/paper/2203.14668/) (2022)
- [Customizing 360-Degree Panoramas through Text-to-Image Diffusion Models](https://mlsanity.com/paper/2310.18840/) (2023)

## Abstract (arXiv)

We advance the field of HDR environment map estimation from a single-view image by establishing a novel approach leveraging the Latent Diffusion Model (LDM) to produce high-quality environment maps that can plausibly light mirror-reflective surfaces. A common issue when using the ERP representation, the format used by the vast majority of approaches, is distortions at the poles and a seam at the sides of the environment map. We remove the border seam artefact by proposing an ERP convolutional padding in the latent autoencoder. Additionally, we investigate whether adapting the diffusion network architecture to the ERP format can improve the quality and accuracy of the estimated environment map by proposing a panoramically-adapted Diffusion Transformer architecture. Our proposed PanoDiT network reduces ERP distortions and artefacts, but at the cost of image quality and plausibility. We evaluate with standard benchmarks to demonstrate that our models estimate high-quality environment maps that perform competitively with state-of-the-art approaches in both image quality and lighting accuracy.