mlsanityData to 14 Oct 2026

Exploring Sparse MoE in GANs for Text-conditioned Image Synthesis

arXiv 2309.03904 ↗First posted 7 Sep 2023cs.CVcited by 13

On the map: Mixture-of-experts → Vision mixture-of-experts

Abstract (arXiv)

Due to the difficulty in scaling up, generative adversarial networks (GANs) seem to be falling from grace on the task of text-conditioned image synthesis. Sparsely-activated mixture-of-experts (MoE) has recently been demonstrated as a valid solution to training large-scale models with limited computational resources. Inspired by such a philosophy, we present Aurora, a GAN-based text-to-image generator that employs a collection of experts to learn feature processing, together with a sparse router to help select the most suitable expert for each feature point. To faithfully decode the sampling stochasticity and the text condition to the final synthesis, our router adaptively makes its decision by taking into account the text-integrated global latent code. At 64x64 image resolution, our model trained on LAION2B-en and COYO-700M achieves 6.2 zero-shot FID on MS COCO. We release the code and checkpoints to facilitate the community for further development.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in Vision mixture-of-experts, one of 12 directions in Mixture-of-experts. Builds on 86 of its 115 references that are in the map. First posted on arXiv 7 Sep 2023.

Builds on

86 of 115 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Scaling Vision-Language Models with Sparse Mixture of Experts13 Mar
  2. 2021Scaling Vision with Sparse Mixture of Experts10 Jun

Other references · 84

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  6. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  7. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  8. 2020Denoising Diffusion Implicit Modelscited by 9,781
  9. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  10. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  11. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  12. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
38 more
  1. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  2. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  3. 2017Neural Discrete Representation Learningcited by 4,712
  4. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  5. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  6. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  7. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  8. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  9. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  10. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  11. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  12. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  13. 2016Improved Techniques for Training GANscited by 2,970
  14. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  15. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  16. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  17. 2014Conditional Generative Adversarial Netscited by 2,162
  18. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
  19. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,979
  20. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,908
  21. 2023Consistency Modelscited by 1,822
  22. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,636
  23. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
  24. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  25. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  26. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  27. 2021Alias-Free Generative Adversarial Networkscited by 1,184
  28. 2016Least Squares Generative Adversarial Networkscited by 1,001
  29. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 931
  30. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
  31. 2016Conditional Image Generation with PixelCNN Decoderscited by 862
  32. 2018Self-Attention Generative Adversarial Networkscited by 809
  33. 2016Generative Adversarial Text to Image Synthesiscited by 735
  34. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  35. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 694
  36. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 689
  37. 2018Which Training Methods for GANs do actually Converge?cited by 679
  38. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 648

50 most cited of 84 All 84 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 6; 2026: 4.

13 papers.

Sort:

By area:

  1. 2024GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generationcited by 301
  2. 2025X-Fusion: Introducing New Modality to Frozen Large Language Modelscited by 13
  3. 2025Ideas in Inference-time Scaling can Benefit Generative Pre-training Algorithmscited by 12
  4. 2025Adversarial Flow Modelscited by 12
  5. 2023SMaRt: Improving GANs with Score Matching Regularitycited by 6
  6. 2024Multi-View Large Reconstruction Model via Geometry-Aware Positional Encoding and Attentioncited by 6
  7. 2026LMMs Meet Object-Centric Vision: Understanding, Segmentation, Editing and Generationcited by 6
  8. 2025Scalable GANs with Transformerscited by 5
  9. 2026TP-Blend: Textual-Prompt Attention Pairing for Precise Object-Style Blending in Diffusion Modelscited by 4
  10. 2025Towards Scalable Topological Regularizerscited by 1
  11. 2025UCD: Unconditional Discriminator Promotes Nash Equilibrium in GANscited by 0
  12. 2026A Replicate-and-Quantize Strategy for Plug-and-Play Load Balancing of Sparse Mixture-of-Experts LLMscited by 0
1 more
  1. 2026Cross-scale Aligned Supervision for Training GANscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Nucleus-Image: Sparse MoE for Image Generation
  2. 2023Improved Training of Mixture-of-Experts Language GANs
  3. 2018MEGAN: Mixture of Experts of Generative Adversarial Networks for Multimodal Image Generation
  4. 2024E2GAN: Efficient Training of Efficient GANs for Image-to-Image Translation
  5. 2023Mobile V-MoEs: Scaling Down Vision Transformers via Sparse Mixture-of-Experts
  6. 2023Robust Mixture-of-Expert Training for Convolutional Neural Networks
  7. 2025Generative Adversarial Networks Bridging Art and Machine Intelligence
  8. 2024Text-to-Image GAN with Pretrained Representations