mlsanityData to 14 Oct 2026

Shadow Generation for Composite Image Using Diffusion model

arXiv 2403.15234 ↗First posted 22 Mar 2024cs.CVcited by 17

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

In the realm of image composition, generating realistic shadow for the inserted foreground remains a formidable challenge. Previous works have developed image-to-image translation models which are trained on paired training data. However, they are struggling to generate shadows with accurate shapes and intensities, hindered by data scarcity and inherent task complexity. In this paper, we resort to foundation model with rich prior knowledge of natural shadow images. Specifically, we first adapt ControlNet to our task and then propose intensity modulation modules to improve the shadow intensity. Moreover, we extend the small-scale DESOBA dataset to DESOBAv2 using a novel data acquisition pipeline. Experimental results on both DESOBA and DESOBAv2 datasets as well as real composite images demonstrate the superior capability of our model for shadow generation task. The dataset, code, and model are released at https://github.com/bcmi/Object-Shadow-Generation-Dataset-DESOBAv2.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 40 of its 58 references that are in the map. First posted on arXiv 22 Mar 2024.

Builds on

40 of 58 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Uni-paint: A Unified Framework for Multimodal Image Inpainting with Pretrained Diffusion Model11 Oct
  2. 2022ShadowDiffusion: When Degradation Prior Meets Diffusion Model for Shadow Removal9 Dec

Other references · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  6. 2020Denoising Diffusion Implicit Modelscited by 9,781
  7. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  8. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,292
  9. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
  10. 2018The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scalecited by 544
  11. 2018Image Inpainting for Irregular Holes Using Partial Convolutionscited by 505
  12. 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
26 more
  1. 2019Pluralistic Image Completioncited by 162
  2. 2019DoveNet: Deep Image Harmonization via Domain Verificationcited by 135
  3. 2017Deep Image Harmonizationcited by 120
  4. 2019Mask-ShadowGAN: Learning to Remove Shadows from Unpaired Datacited by 92
  5. 2018ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositingcited by 90
  6. 2019Deep Parametric Indoor Lighting Estimationcited by 83
  7. 2019Improving the Harmony of the Composite Image by Spatial-Separated Attention Modulecited by 73
  8. 2017GP-GAN: Towards Realistic High-Resolution Image Blendingcited by 64
  9. 2021Making Images Real Again: A Comprehensive Survey on Deep Image Compositioncited by 63
  10. 2019Deep Image Blendingcited by 62
  11. 2021High-Resolution Image Harmonization via Collaborative Dual Transformationscited by 55
  12. 2020BargainNet: Background-Guided Domain Translation for Image Harmonizationcited by 52
  13. 2022CM-GAN: Image Inpainting with Cascaded Modulation GAN and Object-Aware Trainingcited by 49
  14. 2023ControlCom: Controllable Image Composition using Diffusion Modelcited by 43
  15. 2019All-Weather Deep Outdoor Lighting Estimationcited by 41
  16. 2019Automatic Scene Inference for 3D Object Compositingcited by 36
  17. 2022ObjectStitch: Generative Object Compositingcited by 36
  18. 2021Shadow Generation for Composite Image in Real-world Scenescited by 35
  19. 2023ShadowFormer: Global Context Helps Image Shadow Removalcited by 35
  20. 2020SSN: Soft Shadow Network for Image Compositingcited by 26
  21. 2022Instance Shadow Detection with A Single-Stage Detectorcited by 22
  22. 2022Controllable Shadow Generation Using Pixel Height Mapscited by 22
  23. 2020Deep Image Compositingcited by 21
  24. 2023PixHt-Lab: Pixel Height Based Light Effect Generation for Image Compositingcited by 15
  25. 2020Towards Realistic 3D Embedding via View Alignmentcited by 10
  26. 2023Shadow Generation with Decomposed Mask Prediction and Attentive Shadow Fillingcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 7; 2026: 7.

17 papers.

Sort:

By area:

  1. 2026EffectErase: Joint Video Object Removal and Insertion for High-Quality Effect Erasingcited by 15
  2. 2025Precise Object and Effect Removal with Adaptive Target-Aware Attentioncited by 13
  3. 2025MV-CoLight: Efficient Object Compositing with Consistent Lighting and Shadow Generationcited by 8
  4. 2024Unveiling Deep Shadows: A Survey and Benchmark on Image and Video Shadow Detection, Removal, and Generation in the Deep Learning Eracited by 7
  5. 2024Diffree: Text-Guided Shape Free Object Inpainting with Diffusion Modelcited by 6
  6. 2024MetaShadow: Object-Centered Shadow Detection, Removal, and Synthesiscited by 5
  7. 2026Breaking Dual Bottlenecks: Evolving Unified Multimodal Models into Self-Adaptive Interleaved Visual Reasonerscited by 3
  8. 2025Prompt-Aware Controllable Shadow Removalcited by 2
  9. 2025Refracting Reality: Generating Images with Realistic Transparent Objectscited by 2
  10. 2025Preserve Anything: Controllable Image Synthesis with Object Preservationcited by 1
  11. 2025FastInstShadow: A Simple Query-Based Model for Instance Shadow Detectioncited by 0
  12. 2025Embedding Physical Reasoning into Diffusion-Based Shadow Generation Under the Sun and Skycited by 0
5 more
  1. 2026Joint Shadow Generation and Relighting via Light-Geometry Interaction Mapscited by 0
  2. 2026MultiShadow: Multi-Object Shadow Generation for Image Compositing via Diffusion Modelcited by 0
  3. 2026VSDiffusion: Taming Ill-Posed Shadow Generation via Visibility-Constrained Diffusioncited by 0
  4. 2026Reflection Generation for Composite Image Using Diffusion Modelcited by 0
  5. 2026PlantShade: Predicting Plant Shadows for Lighting-Aware Robotic Agricultural Operationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Controllable Shadow Generation with Single-Step Diffusion Models from Synthetic Data
  2. 2023Latent Feature-Guided Diffusion Models for Shadow Removal
  3. 2025Parametric Shadow Control for Portrait Generation in Text-to-Image Diffusion Models
  4. 2024Generative Portrait Shadow Removal
  5. 2024Diff-Shadow: Global-guided Diffusion Model for Shadow Removal
  6. 2024Detail-Preserving Latent Diffusion for Stable Shadow Removal
  7. 2024Controlling the Latent Diffusion Model for Generative Image Shadow Removal via Residual Generation
  8. 2024SpotLight: Shadow-Guided Object Relighting via Diffusion