mlsanityData to 7 Oct 2026

Latent Feature-Guided Diffusion Models for Shadow Removal

arXiv 2312.02156 ↗First posted 4 Dec 2023cs.CV cs.AIcited by 28

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Recovering textures under shadows has remained a challenging problem due to the difficulty of inferring shadow-free scenes from shadow images. In this paper, we propose the use of diffusion models as they offer a promising approach to gradually refine the details of shadow regions during the diffusion process. Our method improves this process by conditioning on a learned latent feature space that inherits the characteristics of shadow-free images, thus avoiding the limitation of conventional methods that condition on degraded images only. Additionally, we propose to alleviate potential local optima during training by fusing noise features with the diffusion network. We demonstrate the effectiveness of our approach which outperforms the previous best method by 13% in terms of RMSE on the AISTD dataset. Further, we explore instance-level shadow removal, where our model outperforms the previous best method by 82% in terms of RMSE on the DESOBA dataset.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 43 of its 58 references that are in the map. First posted on arXiv 4 Dec 2023.

Builds on

43 of 58 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022ShadowDiffusion: When Degradation Prior Meets Diffusion Model for Shadow Removal9 Dec
  2. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Models24 Jan

Other references · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  9. 2022Classifier-Free Diffusion Guidancecited by 5,717
  10. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  11. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  12. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
29 more
  1. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  2. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representationcited by 447
  3. 2017Wasserstein Auto-Encoderscited by 371
  4. 2022Perception Prioritized Training of Diffusion Modelscited by 301
  5. 2019Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishingcited by 191
  6. 2017Stacked Conditional Generative Adversarial Networks for Jointly Learning Shadow Detection and Shadow Removalcited by 160
  7. 2017InfoVAE: Information Maximizing Variational Autoencoderscited by 124
  8. 2019Lagging Inference Networks and Posterior Collapse in Variational Autoencoderscited by 124
  9. 2019Towards Ghost-free Shadow Removal via Dual Hierarchical Aggregation Network and Shadow Matting GANcited by 102
  10. 2019Shadow Removal via Shadow Image Decompositioncited by 96
  11. 2019Mask-ShadowGAN: Learning to Remove Shadows from Unpaired Datacited by 91
  12. 2017Improved Variational Autoencoders for Text Modeling using Dilated Convolutionscited by 81
  13. 2021Auto-Exposure Fusion for Single-Image Shadow Removalcited by 78
  14. 2022DC-ShadowNet: Single-Image Hard and Soft Shadow Removal Using Unsupervised Domain-Classifier Guided Networkcited by 68
  15. 2020From Shadow Segmentation to Shadow Removalcited by 59
  16. 2021From Shadow Generation to Shadow Removalcited by 54
  17. 2019ARGAN: Attentive Recurrent Generative Adversarial Network for Shadow Detection and Removalcited by 51
  18. 2018Avoiding Latent Variable Collapse With Generative Skip Modelscited by 47
  19. 2021CANet: A Context-Aware Network for Shadow Removalcited by 47
  20. 2022Face Relighting with Geometrically Consistent Shadowscited by 43
  21. 2019Instance Shadow Detectioncited by 37
  22. 2019RIS-GAN: Explore Residual and Illumination with Generative Adversarial Networks for Shadow Removalcited by 35
  23. 2021Shadow Generation for Composite Image in Real-world Scenescited by 35
  24. 2023ShadowFormer: Global Context Helps Image Shadow Removalcited by 35
  25. 2017Improving Shadow Suppression for Illumination Robust Face Recognitioncited by 29
  26. 2021Learning from Synthetic Shadows for Shadow Detection and Removalcited by 26
  27. 2022Knowledge Distillation of Transformer-based Language Models Revisitedcited by 11
  28. 2020No Shadow Left Behind: Removing Objects and their Shadows using Approximate Lighting and Geometrycited by 9
  29. 2022CRFormer: A Cross-Region Transformer for Shadow Removalcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 6; 2026: 9.

28 papers.

Sort:

By area:

  1. 2024ObjectDrop: Bootstrapping Counterfactuals for Photorealistic Object Removal and Insertioncited by 71
  2. 2025The Power of Context: How Multimodality Improves Image Super-Resolutioncited by 15
  3. 2024Detail-Preserving Latent Diffusion for Stable Shadow Removalcited by 13
  4. 2024Single-Image Shadow Removal Using Deep Learning: A Comprehensive Surveycited by 7
  5. 2024S3R-Net: A Single-Stage Approach to Self-Supervised Shadow Removalcited by 6
  6. 2024Unveiling Deep Shadows: A Survey and Benchmark on Image and Video Shadow Detection, Removal, and Generation in the Deep Learning Eracited by 6
  7. 2024Shadow Removal Refinement via Material-Consistent Shadow Edgescited by 5
  8. 2025DenseSR: Image Shadow Removal as Dense Predictioncited by 5
  9. 2024FieldNet: Efficient Real-Time Shadow Removal for Enhanced Vision in Field Roboticscited by 4
  10. 2024ShadowHack: Hacking Shadows via Luminance-Color Divide and Conquercited by 4
  11. 2026PhaSR: Generalized Image Shadow Removal with Physically Aligned Priorscited by 4
  12. 2024Semantic-guided Adversarial Diffusion Model for Self-supervised Shadow Removalcited by 3
16 more
  1. 2024Pre-training Feature Guided Diffusion Model for Speech Enhancementcited by 2
  2. 2025Prompt-Aware Controllable Shadow Removalcited by 2
  3. 2026DINOLight: Robust Ambient Light Normalization with Self-supervised Visual Prior Integrationcited by 2
  4. 2024ShadowMamba: State-Space Model with Boundary-Region Selective Scan for Shadow Removalcited by 1
  5. 2024Controlling the Latent Diffusion Model for Generative Image Shadow Removal via Residual Generationcited by 1
  6. 2026On the Global Photometric Alignment for Low-Level Visioncited by 1
  7. 2024Soft-Hard Attention U-Net Model and Benchmark Dataset for Multiscale Image Shadow Removalcited by 0
  8. 2025Shadow Erosion and Nighttime Adaptability for Camera-Based Automated Driving Applicationscited by 0
  9. 2025Contrast-Prior Enhanced Duality for Mask-Free Shadow Removalcited by 0
  10. 2025Generative Manifold Distillation: Aligning Restoration Trajectories with Natural Image Priorcited by 0
  11. 2026CANDLE: Illumination-Invariant Semantic Priors for Color Ambient Lighting Normalizationcited by 0
  12. 2026OmniLight: One Model to Rule All Lighting Conditionscited by 0
  13. 2026Winner of CVPR2026 NTIRE Challenge on Image Shadow Removal: Semantic and Geometric Guidance for Shadow Removal via Cascaded Refinementcited by 0
  14. 2026WildShadowRemover: In-the-Wild Video Shadow Removal via Detail-Preserving Video Diffusion Modelscited by 0
  15. 2026FreeShadow: Training-Free Shadow Removal via Illumination Transfer and Selective Content Preservation in Diffusion Modelscited by 0
  16. 2026Consistency as Regularization for Unsupervised Shadow Removalcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Diff-Shadow: Global-guided Diffusion Model for Shadow Removal
  2. 2025DocShaDiffusion: Diffusion Model in Latent Space for Document Image Shadow Removal
  3. 2025Structure-Guided Diffusion Models for High-Fidelity Portrait Shadow Removal
  4. 2026ODDR: One-Step Deshadow Diffusion via Reward Guidance
  5. 2024Generative Portrait Shadow Removal
  6. 2024Diffusion Models in Low-Level Vision: A Survey
  7. 2023Refusion: Enabling Large-Size Realistic Image Restoration with Latent-Space Diffusion Models
  8. 2023Masked Diffusion Models Are Fast Distribution Learners