mlsanityData to 7 Oct 2026

HDR Environment Map Estimation for Real-Time Augmented Reality

arXiv 2011.10687 ↗First posted 21 Nov 2020cs.CV cs.AI cs.GRcited by 46

Not on the map. Why?

Abstract (arXiv)

We present a method to estimate an HDR environment map from a narrow field-of-view LDR camera image in real-time. This enables perceptually appealing reflections and shading on virtual objects of any material finish, from mirror to diffuse, rendered into a real physical environment using augmented reality. Our method is based on our efficient convolutional neural network architecture, EnvMapNet, trained end-to-end with two novel losses, ProjectionLoss for the generated image, and ClusterLoss for adversarial training. Through qualitative and quantitative comparison to state-of-the-art methods, we demonstrate that our algorithm reduces the directional error of estimated light sources by more than 50%, and achieves 3.7 times lower Frechet Inception Distance (FID). We further showcase a mobile application that is able to run our neural network model in under 9 ms on an iPhone XS, and render in real-time, visually coherent virtual objects in previously unseen real-world environments.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 8; 2023: 13; 2024: 6; 2025: 15; 2026: 4.

46 papers.

Sort:

By area:

  1. 2024Editable Scene Simulation for Autonomous Driving via Collaborative LLM-Agentscited by 91
  2. 2023DiffusionLight: Light Probes for Free by Painting a Chrome Ballcited by 63
  3. 2023PanoDiffusion: 360-degree Panorama Outpainting via Diffusioncited by 50
  4. 2022Diverse Plausible 360-Degree Image Outpainting for Efficient 3DCG Background Creationcited by 47
  5. 2022Neural Light Field Estimation for Street Scenes with Differentiable Virtual Object Insertioncited by 35
  6. 2025CubeDiff: Repurposing Diffusion-Based Image Models for Panorama Generationcited by 35
  7. 2023Autoregressive Omni-Aware Outpainting for Open-Vocabulary 360-Degree Image Generationcited by 33
  8. 2022Guided Co-Modulated GAN for 360deg Field of View Extrapolationcited by 32
  9. 2023EverLight: Indoor-Outdoor Editable HDR Lighting Estimationcited by 32
  10. 20253D Scene Generation: A Surveycited by 24
  11. 2025A Survey of Representation Learning, Optimization Strategies, and Applications for Omnidirectional Visioncited by 23
  12. 2023LightSim: Neural Lighting Simulation for Urban Scenescited by 20
34 more
  1. 2023NeRRF: 3D Reconstruction and View Synthesis for Transparent and Specular Objects with Neural Refractive-Reflective Fieldscited by 16
  2. 2022Cylin-Painting: Seamless 360textdegree Panoramic Image Outpainting and Beyondcited by 12
  3. 2022Editable Indoor Lighting Estimationcited by 12
  4. 2023Spatiotemporally Consistent HDR Indoor Lighting Estimationcited by 12
  5. 2025DreamCube: 3D Panorama Generation via Multi-plane Synchronizationcited by 12
  6. 2023The Sky's the Limit: Re-lightable Outdoor Scenes via a Sky-pixel Constrained Illumination Prior and Outside-In Visibilitycited by 11
  7. 2025GaSLight: Gaussian Splats for Spatially-Varying Lighting in HDRcited by 11
  8. 2025DiffusionLight-Turbo: Accelerated Light Probes for Free via Single-Pass Chrome Ball Inpaintingcited by 11
  9. 2024OPa-Ma: Text Guided Mamba for 360-degree Image Out-paintingcited by 7
  10. 2026360Anything: Geometry-Free Lifting of Images and Videos to 360degcited by 7
  11. 2023RENI++ A Rotation-Equivariant, Scale-Invariant, Natural Illumination Priorcited by 6
  12. 2024CamFreeDiff: Camera-free Image to Panorama Generation with Diffusion Modelcited by 5
  13. 2025Spatiotemporally Consistent Indoor Lighting Estimation with Diffusion Priorscited by 5
  14. 2022HORIZON: High-Resolution Semantically Controlled Panorama Synthesiscited by 3
  15. 2023Multi-Camera Lighting Estimation for Photorealistic Front-Facing Mobile Augmented Realitycited by 3
  16. 2023Local-to-Global Panorama Inpainting for Locale-Aware Indoor Lighting Predictioncited by 3
  17. 2024360U-Former: HDR Illumination Estimation with Panoramic Adapted Vision Transformerscited by 3
  18. 2024CleAR: Robust Context-Guided Generative Lighting Estimation for Mobile Augmented Realitycited by 3
  19. 2022Real-time Virtual-Try-On from a Single Example Image through Deep Inverse Graphics and Learned Differentiable Rendererscited by 2
  20. 2023LitAR: Visually Coherent Lighting for Mobile Augmented Realitycited by 1
  21. 2025ComGS: Efficient 3D Object-Scene Composition via Surface Octahedral Probescited by 1
  22. 2025Generative Augmented Reality: Paradigms, Technologies, and Future Applicationscited by 1
  23. 2022Hair Color Digitization through Imaging and Deep Inverse Graphicscited by 0
  24. 2023Real-time Light Estimation and Neural Soft Shadows for AR Indoor Scenarioscited by 0
  25. 2024Towards Physically-Based Sky-Modelingcited by 0
  26. 2025HDR Environment Map Estimation with Latent Diffusion Modelscited by 0
  27. 2025AR as an Evaluation Playground: Bridging Metrics and Visual Perception of Computer Vision Modelscited by 0
  28. 2025Improving the color accuracy of lighting estimation modelscited by 0
  29. 2025Embedding Physical Reasoning into Diffusion-Based Shadow Generation Under the Sun and Skycited by 0
  30. 2025Cast and Attached Shadow Detection via Iterative Light and Geometry Reasoningcited by 0
  31. 2025Towards Physically-Based Sky-Modeling For Image Based Lightingcited by 0
  32. 2026RehearsalNeRF: Decoupling Intrinsic Neural Fields of Dynamic Illuminations for Scene Editingcited by 0
  33. 2026LatentHDR: Decoupling Exposure from Diffusion via Conditional Latent-to-Latent Mapping for Text/Image-to-Panoramic HDRcited by 0
  34. 2026PanoLess: Environment Reconstruction from Partial Reflective Viewscited by 0