mlsanityData to 7 Oct 2026

Mitigating Memorization in Text-to-Image Diffusion via Region-Aware Prompt Augmentation and Multimodal Copy Detection

arXiv 2603.13070 ↗First posted 13 Mar 2026cs.CVcited by 0

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

State-of-the-art text-to-image diffusion models can produce impressive visuals but may memorize and reproduce training images, creating copyright and privacy risks. Existing prompt perturbations applied at inference time, such as random token insertion or embedding noise, may lower copying but often harm image-prompt alignment and overall fidelity. To address this, we introduce two complementary methods. First, Region-Aware Prompt Augmentation (RAPTA) uses an object detector to find salient regions and turn them into semantically grounded prompt variants, which are randomly sampled during training to increase diversity, while maintaining semantic alignment. Second, Attention-Driven Multimodal Copy Detection (ADMCD) aggregates local patch, global semantic, and texture cues with a lightweight transformer to produce a fused representation, and applies simple thresholded decision rules to detect copying without training with large annotated datasets. Experiments show that RAPTA reduces overfitting while maintaining high synthesis quality, and that ADMCD reliably detects copying, outperforming single-modal metrics.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 14 of its 21 references that are in the map. First posted on arXiv 13 Mar 2026.

Builds on

14 of 21 references.

Closest ancestors · 2

Same direction, newest first

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  2. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 12

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  7. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  8. 2020Image Quality Assessment: Unifying Structure and Texture Similaritycited by 792
  9. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 774
  10. 2023Extracting Training Data from Diffusion Modelscited by 723
  11. 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 418
  12. 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 322

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Rethinking Memorization Mitigation in Diffusion Models: Reinforcing Text Conditioning
  2. 2024Unveiling and Mitigating Memorization in Text-to-image Diffusion Models through Cross Attention
  3. 2024Exploring Local Memorization in Diffusion Models via Bright Ending Attention
  4. 2024Memorized Images in Diffusion Models share a Subspace that can be Located and Deleted
  5. 2023Understanding and Mitigating Copying in Diffusion Models
  6. 2025Finding DoRI: Discovery of Retained Images in Diffusion Models
  7. 2026Prompt Reinjection: Alleviating Prompt Forgetting in Multimodal Diffusion Transformers for Text-to-Image Generation
  8. 2024Detecting, Explaining, and Mitigating Memorization in Diffusion Models