# Good Seed Makes a Good Crop: Discovering Secret Seeds in Text-to-Image Diffusion Models

arXiv: https://arxiv.org/abs/2405.14828 · HTML: https://mlsanity.com/paper/2405.14828/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 38 of its 58 references that are in the map.
- First posted on arXiv 23 May 2024.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 38)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022)
- [eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers](https://mlsanity.com/paper/2211.01324/) (2022)
- [Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation](https://mlsanity.com/paper/2211.12572/) (2022)
- [Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis](https://mlsanity.com/paper/2306.09341/) (2023)
- [PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis](https://mlsanity.com/paper/2310.00426/) (2023)
- [Adversarial Diffusion Distillation](https://mlsanity.com/paper/2311.17042/) (2023)
- [The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale](https://mlsanity.com/paper/1811.00982/) (2018)
- [Diffusion Self-Guidance for Controllable Image Generation](https://mlsanity.com/paper/2306.00986/) (2023)
- [DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editing](https://mlsanity.com/paper/2306.14435/) (2023)
- [EfficientFormer: Vision Transformers at MobileNet Speed](https://mlsanity.com/paper/2206.01191/) (2022)
- [Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs](https://mlsanity.com/paper/2401.11708/) (2024)
- [Accounting for Variance in Machine Learning Benchmarks](https://mlsanity.com/paper/2103.03098/) (2021)
- [FreeU: Free Lunch in Diffusion U-Net](https://mlsanity.com/paper/2309.11497/) (2023)
- [Localizing Object-level Shape Variations with Text-to-Image Diffusion Models](https://mlsanity.com/paper/2303.11306/) (2023)
- [InitNO: Boosting Text-to-Image Diffusion Models via Initial Noise Optimization](https://mlsanity.com/paper/2404.04650/) (2024)
- [Uncovering the Disentanglement Capability in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2212.08698/) (2022)
- [Torch.manual_seed(3407) is all you need: On the influence of random seeds in deep learning architectures for computer vision](https://mlsanity.com/paper/2109.08203/) (2021)
- [Particle Guidance: non-I.I.D. Diverse Sampling with Diffusion Models](https://mlsanity.com/paper/2310.13102/) (2023)
- [Guided Image Synthesis via Initial Image Editing in Diffusion Model](https://mlsanity.com/paper/2305.03382/) (2023)
- [Wuerstchen: An Efficient Architecture for Large-Scale Text-to-Image Diffusion Models](https://mlsanity.com/paper/2306.00637/) (2023)

## Built on by (most cited first, 30 of 48)

- [Golden Noise for Diffusion Models: A Learning Framework](https://mlsanity.com/paper/2411.09502/) (2024)
- [A Noise is Worth Diffusion Guidance](https://mlsanity.com/paper/2412.03895/) (2024)
- [Zigzag Diffusion Sampling: Diffusion Models Can Self-Improve via Self-Reflection](https://mlsanity.com/paper/2412.10891/) (2024)
- [LumiNet: Latent Intrinsics Meets Diffusion Models for Indoor Scene Relighting](https://mlsanity.com/paper/2412.00177/) (2024)
- [Beyond Aesthetics: Cultural Competence in Text-to-Image Models](https://mlsanity.com/paper/2407.06863/) (2024)
- [IMAGHarmony: Controllable Image Editing with Consistent Object Quantity and Layout](https://mlsanity.com/paper/2506.01949/) (2025)
- [ZeroComp: Zero-shot Object Compositing from Image Intrinsics via Diffusion](https://mlsanity.com/paper/2410.08168/) (2024)
- [Breaking Latent Prior Bias in Detectors for Generalizable AIGC Image Detection](https://mlsanity.com/paper/2506.00874/) (2025)
- [The Silent Assistant: NoiseQuery as Implicit Guidance for Goal-Driven Image Generation](https://mlsanity.com/paper/2412.05101/) (2024)
- [ScalingNoise: Scaling Inference-Time Search for Generating Infinite Videos](https://mlsanity.com/paper/2503.16400/) (2025)
- [TKG-DM: Training-free Chroma Key Content Generation Diffusion Model](https://mlsanity.com/paper/2411.15580/) (2024)
- [Navigating with Annealing Guidance Scale in Diffusion Space](https://mlsanity.com/paper/2506.24108/) (2025)
- [It's Never Too Late: Noise Optimization for Collapse Recovery in Trained Diffusion Models](https://mlsanity.com/paper/2601.00090/) (2025)
- [Model Already Knows the Best Noise: Bayesian Active Noise Selection via Attention in Video Diffusion Model](https://mlsanity.com/paper/2505.17561/) (2025)
- [NoiseAR: AutoRegressing Initial Noise Prior for Diffusion Models](https://mlsanity.com/paper/2506.01337/) (2025)
- [Adjusting Initial Noise to Mitigate Memorization in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2510.08625/) (2025)
- [RAGME: Retrieval Augmented Video Generation for Enhanced Motion Realism](https://mlsanity.com/paper/2504.06672/) (2025)
- [Early Timestep Zero-Shot Candidate Selection for Instruction-Guided Image Editing](https://mlsanity.com/paper/2504.13490/) (2025)
- [GenTune: Toward Traceable Prompts to Improve Controllability of Image Refinement in Environment Design](https://mlsanity.com/paper/2508.15227/) (2025)
- [Questionable practices in machine learning](https://mlsanity.com/paper/2407.12220/) (2024)
- [Early Estimation of Language to Latent Alignment in Diffusion Models](https://mlsanity.com/paper/2512.08505/) (2025)
- [FastInit: Fast Noise Initialization for Temporally Consistent Video Generation](https://mlsanity.com/paper/2506.16119/) (2025)
- [TAUE: Training-free Noise Transplant and Cultivation Diffusion Model](https://mlsanity.com/paper/2511.02580/) (2025)
- [RSEdit: Text-Guided Image Editing for Remote Sensing](https://mlsanity.com/paper/2603.13708/) (2026)
- [Couple to Control: Joint Initial Noise Design in Diffusion Models](https://mlsanity.com/paper/2605.11311/) (2026)
- [Colored Noise Diffusion Sampling](https://mlsanity.com/paper/2605.30332/) (2026)
- [The FID Lottery: Quantifying Hidden Randomness in Generative-Model Evaluation](https://mlsanity.com/paper/2606.20536/) (2026)
- [PhysWeep: Does a Video Generator Realize the Physics You Ask For?](https://mlsanity.com/paper/2609.06207/) (2026)
- [Language-based Image Colorization: A Benchmark and Beyond](https://mlsanity.com/paper/2503.14974/) (2025)
- [Ctrl-Z Sampling: Scaling Diffusion Sampling with Controlled Random Zigzag Explorations](https://mlsanity.com/paper/2506.20294/) (2025)

## Abstract (arXiv)

Recent advances in text-to-image (T2I) diffusion models have facilitated creative and photorealistic image synthesis. By varying the random seeds, we can generate many images for a fixed text prompt. Technically, the seed controls the initial noise and, in multi-step diffusion inference, the noise used for reparameterization at intermediate timesteps in the reverse diffusion process. However, the specific impact of the random seed on the generated images remains relatively unexplored. In this work, we conduct a large-scale scientific study into the impact of random seeds during diffusion inference. Remarkably, we reveal that the best 'golden' seed achieved an impressive FID of 21.60, compared to the worst 'inferior' seed's FID of 31.97. Additionally, a classifier can predict the seed number used to generate an image with over 99.9% accuracy in just a few epochs, establishing that seeds are highly distinguishable based on generated images. Encouraged by these findings, we examined the influence of seeds on interpretable visual dimensions. We find that certain seeds consistently produce grayscale images, prominent sky regions, or image borders. Seeds also affect image composition, including object location, size, and depth. Moreover, by leveraging these 'golden' seeds, we demonstrate improved image generation such as high-fidelity inference and diversified sampling. Our investigation extends to inpainting tasks, where we uncover some seeds that tend to insert unwanted text artifacts. Overall, our extensive analyses highlight the importance of selecting good seeds and offer practical utility for image generation.