mlsanityData to 14 Oct 2026

It's Never Too Late: Noise Optimization for Collapse Recovery in Trained Diffusion Models

arXiv 2601.00090 ↗First posted 31 Dec 2025cs.CV cs.LGcited by 11

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Contemporary text-to-image models exhibit a surprising degree of mode collapse, as can be seen when sampling several images given the same text prompt. Previous work has attempted to address this issue by steering the model using guidance mechanisms, or by generating a large pool of candidates and refining them. In this work, we take a different direction and aim for diversity in generations via noise optimization. Specifically, we show that a simple noise optimization objective can mitigate mode collapse while preserving the fidelity of the base model. We also analyze the frequency characteristics of the noise and show that alternative noise initializations with different frequency profiles can improve both optimization and search. Our experiments demonstrate that noise optimization yields superior results in terms of generation quality and diversity.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 67 of its 84 references that are in the map. First posted on arXiv 31 Dec 2025.

Builds on

67 of 84 references.

Closest ancestors · 20

Same direction, newest first

  1. 2025TCFG: Tangential Damping Classifier-free Guidance23 Mar
  2. 2025Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Steps16 Jan
  3. 2025Inference-Time Alignment in Diffusion Models with Reward-Guided Generation: Tutorial and Review16 Jan
  4. 2024A Noise is Worth Diffusion Guidance5 Dec
  5. 2024Shielded Diffusion: Generating Novel and Diverse Images using Sparse Repellency8 Oct
  6. 2024Eliminating Oversaturation and Artifacts of High Guidance Scales in Diffusion Models3 Oct
  7. 2024CFG++: Manifold-constrained Classifier Free Guidance for Diffusion Models12 Jun
  8. 2024Guiding a Diffusion Model with a Bad Version of Itself4 Jun
  9. 2024Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Models11 Apr
  10. 2024Self-Rectifying Diffusion Sampling with Perturbed-Attention Guidance26 Mar
  11. 2023Adversarial Diffusion Distillation28 Nov
  12. 2023CADS: Unleashing the Diversity of Diffusion Models through Condition-Annealed Sampling26 Oct
8 more
  1. 2023Particle Guidance: non-I.I.D. Diverse Sampling with Diffusion Models19 Oct
  2. 2023End-to-End Diffusion Latent Optimization Improves Classifier Guidance23 Mar
  3. 2022Classifier-Free Diffusion Guidance26 Jul
  4. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  5. 2021Variational Diffusion Models1 Jul
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  7. 2020Denoising Diffusion Implicit Models6 Oct
  8. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 47

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  6. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  7. 2022Flow Matching for Generative Modelingcited by 5,702
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  9. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  10. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  11. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  12. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
35 more
  1. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
  2. 2018Demystifying MMD GANscited by 1,293
  3. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,179
  4. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
  5. 2022Building Normalizing Flows with Stochastic Interpolantscited by 995
  6. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
  7. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  8. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 890
  9. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 475
  10. 2022The Vendi Score: A Diversity Evaluation Metric for Machine Learningcited by 311
  11. 2023HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Modelscited by 225
  12. 2024InitNO: Boosting Text-to-Image Diffusion Models via Initial Noise Optimizationcited by 123
  13. 2024D-Flow: Differentiating through Flows for Controlled Generationcited by 122
  14. 2024ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimizationcited by 119
  15. 2023Optimizing Diffusion Noise Can Serve As Universal Motion Priorscited by 98
  16. 2023FreeInit: Bridging Initialization Gap in Video Diffusion Modelscited by 95
  17. 2024Improving Text-to-Image Consistency via Automatic Prompt Optimizationcited by 90
  18. 2025SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency Distillationcited by 87
  19. 2023Generating images of rare concepts using pre-trained diffusion modelscited by 79
  20. 2024DITTO: Diffusion Inference-Time T-Optimization for Music Generationcited by 73
  21. 2024Inference-Time Alignment of Diffusion Models with Direct Noise Optimizationcited by 55
  22. 2024Good Seed Makes a Good Crop: Discovering Secret Seeds in Text-to-Image Diffusion Modelscited by 48
  23. 2023Norm-guided latent space exploration for text-to-image generationcited by 43
  24. 2023If at First You Don't Succeed, Try, Try Again: Faithful Diffusion-based Text-to-Image Generation by Selectioncited by 40
  25. 2018GDPP: Learning Diverse Generations Using Determinantal Point Processcited by 26
  26. 2025SliderSpace: Decomposing the Visual Capabilities of Diffusion Modelscited by 26
  27. 2025Reward-Guided Iterative Refinement in Diffusion Models at Test-Time with Applications to Protein and DNA Designcited by 25
  28. 2024The Crystal Ball Hypothesis in diffusion models: Anticipating object positions from initial noisecited by 19
  29. 2025DiverseFlow: Sample-Efficient Diverse Mode Coverage in Flowscited by 17
  30. 2025Scaling Group Inference for Diverse and High-Quality Generationcited by 12
  31. 2025Entropy Rectifying Guidance for Diffusion and Flow Modelscited by 11
  32. 2025LLMs can see and hear without any trainingcited by 8
  33. 2024CoCoNO: Attention Contrast-and-Complete for Initial Noise Optimization in Text-to-Image Synthesiscited by 7
  34. 2025Benchmarking Diversity in Image Generation via Attribute-Conditional Human Evaluationcited by 5
  35. 2024Negative Token Merging: Image-based Adversarial Feature Guidancecited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 11.

11 papers.

  1. 2026STRIDE: Training-Free Diversity Guidance via PCA-Directed Feature Perturbation in Single-Step Diffusion Modelscited by 3
  2. 2026StressDream: Steering Video World Models for Robust Policy Evaluation and Improvementcited by 1
  3. 2026Don't Settle at the Mode! Mitigating Diversity Collapse in Pretrained Flow Models via Feature Self-Guidancecited by 1
  4. 2026Manifold-Constrained Noise Optimization for Diverse Diffusion Samplingcited by 1
  5. 2026Diverse Sampling in Diffusion Models with Marginal Preserving Particle Guidancecited by 0
  6. 2026Breaking the Lock-in: Diversifying Text-to-Image Generation via Representation Modulationcited by 0
  7. 2026Anchoring and Steering Diffusion: Enhancing the Faithfulness of Text-to-Image Generation at Inference Timecited by 0
  8. 2026JIVE: Jacobian-Informed Volume Expansion for Diverse Generative Samplingcited by 0
  9. 2026MotionSpaceFlow: Representation-Aware Flow Matching in Direct Motion Spacecited by 0
  10. 2026Traversing the Satisfaction-Diversity Frontier in Text-to-Image Diffusioncited by 0
  11. 2026Controllable Diverse Sampling through Pullback Directionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Diverse Text-to-Image Generation via Contrastive Noise Optimization
  2. 2025A Minimalist Method for Fine-tuning Text-to-Image Diffusion Models
  3. 2026Couple to Control: Joint Initial Noise Design in Diffusion Models
  4. 2024Not All Noises Are Created Equally:Diffusion Noise Selection and Optimization
  5. 2025Noise-Level Diffusion Guidance: Well Begun is Half Done
  6. 2024Controllable Generation with Text-to-Image Diffusion Models: A Survey
  7. 2023On the Importance of Noise Scheduling for Diffusion Models
  8. 2025Fewer Denoising Steps or Cheaper Per-Step Inference: Towards Compute-Optimal Diffusion Model Deployment