mlsanityData to 7 Oct 2026

Breaking the Lock-in: Diversifying Text-to-Image Generation via Representation Modulation

arXiv 2606.06813 ↗First posted 5 Jun 2026cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent text-to-image models built on large-scale Transformer backbones and flow-based objectives deliver strong text-image alignment and high visual quality, yet often produce overly similar samples under a fixed prompt. Existing diversity-enhancement methods alleviate this issue, but typically require expensive sampling or auxiliary optimization, incurring non-trivial overhead. To investigate the root cause of this homogeneity, we examine intermediate Transformer features and observe that the zero-frequency spatial average (DC) component rapidly converges across seeds early in generation, causing early trajectory lock-in that limits downstream variation. Building on this observation, we propose DC Attenuation for diVersity Enhancement (DAVE), a training-free representation-level intervention that selectively attenuates this component in the early regime. DAVE preserves the sampling pipeline with negligible overhead, improving prompt-consistent diversity while maintaining competitive image quality.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 37 of its 43 references that are in the map. First posted on arXiv 5 Jun 2026.

Builds on

37 of 43 references.

References · 37

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2022Flow Matching for Generative Modelingcited by 5,426
  4. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  5. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  6. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  7. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  8. 2018On the Spectral Bias of Neural Networkscited by 1,321
  9. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  10. 2022Diffusion Models in Vision: A Surveycited by 1,008
  11. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 925
  12. 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 722
25 more
  1. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 634
  2. 2020Reliable Fidelity and Diversity Metrics for Generative Modelscited by 425
  3. 2022The Vendi Score: A Diversity Evaluation Metric for Machine Learningcited by 299
  4. 2023P+: Extended Textual Conditioning in Text-to-Image Generationcited by 242
  5. 2023FreeU: Free Lunch in Diffusion U-Netcited by 195
  6. 2025SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformercited by 165
  7. 2023CADS: Unleashing the Diversity of Diffusion Models through Condition-Annealed Samplingcited by 124
  8. 2023Particle Guidance: non-I.I.D. Diverse Sampling with Diffusion Modelscited by 65
  9. 2023Diffusion Models Beat GANs on Image Classificationcited by 57
  10. 2024Consistency-diversity-realism Pareto fronts of conditional image generative modelscited by 35
  11. 2025An Analytical Theory of Spectral Bias in the Learning Dynamics of Diffusion Modelscited by 26
  12. 2024FluxSpace: Disentangled Semantic Editing in Rectified Flow Transformerscited by 25
  13. 2024Shielded Diffusion: Generating Novel and Diverse Images using Sparse Repellencycited by 21
  14. 2024Minority-Focused Text-to-Image Generation via Prompt Optimizationcited by 18
  15. 2025DiverseFlow: Sample-Efficient Diverse Mode Coverage in Flowscited by 17
  16. 2025Exploring Multimodal Diffusion Transformers for Enhanced Prompt-based Image Editingcited by 16
  17. 2025ReFlex: Text-Guided Editing of Real Images in Rectified Flow via Mid-Step Feature Extraction and Attention Adaptationcited by 15
  18. 2025SPARKE: Scalable Prompt-Aware Diversity and Novelty Guidance in Diffusion Models via RKE Scorecited by 14
  19. 2025Enhancing Creative Generation on Stable Diffusion-based Modelscited by 13
  20. 2025It's Never Too Late: Noise Optimization for Collapse Recovery in Trained Diffusion Modelscited by 9
  21. 2025From Navigation to Refinement: Revealing the Two-Stage Nature of Flow-based Diffusion Models through Oracle Velocitycited by 7
  22. 2025Benchmarking Diversity in Image Generation via Attribute-Conditional Human Evaluationcited by 5
  23. 2025SplitFlux: Learning to Decouple Content and Style from a Single Imagecited by 5
  24. 2026TexTailor: Inference-Time Textual Guidance Tailoring for Multimodal Diffusion Transformerscited by 3
  25. 2025How Generative AI supports human in conceptual designcited by 2

Built on by

Not cited yet.