mlsanityData to 14 Oct 2026

EMoE: Training-Free Expert Disagreement for Uncertainty-Aware Text-to-Image Diffusion

arXiv 2505.13273 ↗First posted 19 May 2025cs.AI cs.LGcited by 0

On the map: Diffusion models → Image generation, text-to-image and editing

also in 1 more
Abstract (arXiv)

Large text-to-image diffusion models rarely expose reliable signals of when a prompt is likely to produce a poorly aligned generation, especially when training data is undisclosed. We study whether expert disagreement inside pre-trained mixture-of-experts (MoE) diffusion models can serve as a reliable estimate for epistemic uncertainty. We introduce EMoE, a training-free method that separates expert-specific computation paths at an early MoE layer, uses the same initial noise across paths, and measures variance among their latent representations after the first denoising step. This provides an uncertainty-aware prompt signal before full image generation, without auxiliary networks or training diffusion ensembles. On COCO and CC3M, EMoE ranks prompts by text-image alignment quality metrics more consistently than diffusion-specific and router-based baselines. We further apply EMoE to multilingual prompts and find systematic language-dependent differences in disagreement and generation quality, including shared-vocabulary effects. These results position EMoE as a practical diagnostic tool for prompt risk, model coverage, and bias analysis in MoE text-to-image diffusion models.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 46 of its 60 references that are in the map. First posted on arXiv 19 May 2025.

Builds on

46 of 60 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022Multi-Concept Customization of Text-to-Image Diffusion8 Dec
  2. 2022Imagic: Text-Based Real Image Editing with Diffusion Models17 Oct
  3. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec

Other references · 43

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  6. 2020Denoising Diffusion Implicit Modelscited by 9,781
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  9. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  10. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  11. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  12. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
31 more
  1. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  3. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
  4. 2022Video Diffusion Modelscited by 2,305
  5. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  6. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
  7. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  8. 2023One-step Diffusion with Distribution Matching Distillationcited by 1,019
  9. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 913
  10. 2017Deep Bayesian Active Learning with Image Datacited by 819
  11. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  12. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 670
  13. 2016Deep Exploration via Bootstrapped DQNcited by 527
  14. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 455
  15. 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 432
  16. 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 395
  17. 2020Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learningcited by 249
  18. 2024Arcee's MergeKit: A Toolkit for Merging Large Language Modelscited by 194
  19. 2022Crossmodal-3600: A Massively Multilingual Multimodal Evaluation Datasetcited by 100
  20. 2020Improving Image Captioning with Better Use of Captionscited by 84
  21. 2024Uncertainty Estimation and Quantification for LLMs: A Simple Supervised Approachcited by 76
  22. 2024Estimating Epistemic and Aleatoric Uncertainty with a Single Modelcited by 54
  23. 2023TIBET: Identifying and Evaluating Biases in Text-to-Image Generative Modelscited by 52
  24. 2024CLIP the Bias: How Useful is Balancing Data in Multimodal Learning?cited by 35
  25. 2021Deep Ensembles from a Bayesian Perspectivecited by 27
  26. 2024Shedding Light on Large Generative Networks: Estimating Epistemic Uncertainty in Diffusion Modelscited by 22
  27. 2023Normalizing Flow Ensembles for Rich Aleatoric and Epistemic Uncertainty Modelingcited by 10
  28. 2024Epistemic Uncertainty Quantification For Pre-trained Neural Networkcited by 10
  29. 2024High Frequency Matters: Uncertainty Guided Image Compression with Wavelet Diffusioncited by 4
  30. 2023Training of Neural Networks with Uncertain Data: A Mixture of Experts Approachcited by 2
  31. 2025An Exploration of Default Images in Text-to-Image Generationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Image Generation from Contextually-Contradictory Prompts
  2. 2026Unsure but Certain: Uncovering the Representation-Confidence Gap in Diffusion Language Models
  3. 2024Words Worth a Thousand Pictures: Measuring and Understanding Perceptual Variability in Text-to-Image Generation
  4. 2025Efficient Training of Diffusion Mixture-of-Experts Models: A Practical Recipe
  5. 2026EruDiff: Refactoring Knowledge in Diffusion Models for Advanced Text-to-Image Synthesis
  6. 2025Steer Away From Mode Collisions: Improving Composition In Diffusion Models
  7. 2026ELDiff: When Evidential Learning Meets Text-to-Image Diffusion
  8. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers