mlsanityData to 14 Oct 2026

Text-to-Image Generation via Implicit Visual Guidance and Hypernetwork

arXiv 2208.08493 ↗First posted 17 Aug 2022cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

We develop an approach for text-to-image generation that embraces additional retrieval images, driven by a combination of implicit visual guidance loss and generative objectives. Unlike most existing text-to-image generation methods which merely take the text as input, our method dynamically feeds cross-modal search results into a unified training stage, hence improving the quality, controllability and diversity of generation results. We propose a novel hypernetwork modulated visual-text encoding scheme to predict the weight update of the encoding layer, enabling effective transfer from visual information (e.g. layout, content) into the corresponding latent domain. Experimental results show that our model guided with additional retrieval visual data outperforms existing GAN-based models. On COCO dataset, we achieve better FID of 9.13 with up to 3.5 × fewer generator parameters, compared with the state-of-the-art method.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 37 of its 47 references that are in the map. First posted on arXiv 17 Aug 2022.

Builds on

37 of 47 references.

References · 37

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  6. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  7. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  8. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  10. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  11. 2016Improved Techniques for Training GANscited by 2,970
  12. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
25 more
  1. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  2. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,296
  3. 2016HyperNetworkscited by 1,027
  4. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 991
  5. 2014Deep Visual-Semantic Alignments for Generating Image Descriptionscited by 968
  6. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  7. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  8. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 577
  9. 2019DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesiscited by 308
  10. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  11. 2017StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 229
  12. 2014Unifying Visual-Semantic Embeddings with Multimodal Neural Language Modelscited by 198
  13. 2019Controllable Text-to-Image Generationcited by 187
  14. 2015Learning Deep Structure-Preserving Image-Text Embeddingscited by 139
  15. 2020Adversarial Generation of Continuous Imagescited by 135
  16. 2021Instance-Conditioned GANcited by 121
  17. 2019Object-driven Text-to-Image Synthesis via Adversarial Trainingcited by 117
  18. 2020Image Generators with Conditionally-Independent Pixel Synthesiscited by 106
  19. 2021Stylizing 3D Scene via Implicit Representation and HyperNetworkcited by 103
  20. 2019Semantic Object Accuracy for Generative Text-to-Image Synthesiscited by 89
  21. 2017Look, Imagine and Match: Improving Textual-Visual Cross-Modal Retrieval with Generative Modelscited by 53
  22. 2019Generating Multiple Objects at Spatially Distinct Locationscited by 41
  23. 2020Text-to-Image Generation Grounded by Fine-Grained User Attentioncited by 36
  24. 2016An Empirical Study of Language CNN for Image Captioningcited by 15
  25. 2022Memory-Driven Text-to-Image Generationcited by 10

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 1; 2025: 1.

4 papers.

  1. 2023RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Modelcited by 47
  2. 2024A Survey on Large Language Models from Concept to Implementationcited by 4
  3. 2023DiffBlender: Composable and Versatile Multimodal Text-to-Image Diffusion Modelscited by 1
  4. 2025PartStickers: Generating Parts of Objects for Rapid Prototypingcited by 0