mlsanityData to 14 Oct 2026

Memory-Driven Text-to-Image Generation

arXiv 2208.07022 ↗First posted 15 Aug 2022cs.CV cs.CL cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

We introduce a memory-driven semi-parametric approach to text-to-image generation, which is based on both parametric and non-parametric techniques. The non-parametric component is a memory bank of image features constructed from a training set of images. The parametric component is a generative adversarial network. Given a new text description at inference time, the memory bank is used to selectively retrieve image features that are provided as basic information of target images, which enables the generator to produce realistic synthetic results. We also incorporate the content information into the discriminator, together with semantic features, allowing the discriminator to make a more reliable prediction. Experimental results demonstrate that the proposed memory-driven semi-parametric approach produces more realistic images than purely parametric approaches, in terms of both visual fidelity and text-image semantic consistency.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 41 of its 59 references that are in the map. First posted on arXiv 15 Aug 2022.

Builds on

41 of 59 references.

References · 41

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  6. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  7. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  8. 2016Improved Techniques for Training GANscited by 2,970
  9. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
  10. 2016Pyramid Scene Parsing Networkcited by 2,479
  11. 2014Conditional Generative Adversarial Netscited by 2,162
  12. 2014Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognitioncited by 720
29 more
  1. 2018Which Training Methods for GANs do actually Converge?cited by 679
  2. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  3. 2017Wasserstein GANcited by 639
  4. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 577
  5. 2018Image Generation from Scene Graphscited by 324
  6. 2019DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesiscited by 308
  7. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  8. 2017Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradientscited by 236
  9. 2017StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 229
  10. 2019Controllable Text-to-Image Generationcited by 187
  11. 2014Object Detectors Emerge in Deep Scene CNNscited by 179
  12. 2015ParseNet: Looking Wider to See Bettercited by 177
  13. 2019MirrorGAN: Learning Text-to-image Generation by Redescriptioncited by 176
  14. 2019ManiGAN: Text-Guided Image Manipulationcited by 156
  15. 2019Object-driven Text-to-Image Synthesis via Adversarial Trainingcited by 117
  16. 2018Inferring Semantic Layout for Hierarchical Text-to-Image Synthesiscited by 101
  17. 2018Text-Adaptive Generative Adversarial Networks: Manipulating Images with Natural Languagecited by 90
  18. 2019Semantic Object Accuracy for Generative Text-to-Image Synthesiscited by 89
  19. 2017Semantic Image Synthesis via Adversarial Learningcited by 87
  20. 2019Specifying Object Attributes and Relations in Interactive Scene Generationcited by 76
  21. 2015Learning a Discriminative Model for the Perception of Realism in Composite Imagescited by 52
  22. 2018Semi-parametric Image Synthesiscited by 50
  23. 2019PasteGAN: A Semi-Parametric Method to Generate Image from Scene Graphcited by 44
  24. 2020RetrieveGAN: Image Synthesis via Differentiable Patch Retrievalcited by 35
  25. 2020Lightweight Generative Adversarial Networks for Text-Guided Image Manipulationcited by 33
  26. 2022Word-Level Fine-Grained Story Visualizationcited by 23
  27. 2019Controllable and Progressive Image Extrapolationcited by 13
  28. 2020Image-to-Image Translation with Text Guidancecited by 10
  29. 2020Knowledge Graph Extraction from Videoscited by 10

Built on by

Cited by year

Citing papers per year: 2022: 4; 2023: 3; 2024: 3.

10 papers.

  1. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 354
  2. 2023Text-to-image Diffusion Models in Generative AI: A Surveycited by 265
  3. 2022Re-Imagen: Retrieval-Augmented Text-to-Image Generatorcited by 210
  4. 2023RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Modelcited by 47
  5. 2023Retrieval-Augmented Text-to-Audio Generationcited by 24
  6. 2024ObjectMate: A Recurrence Prior for Object Insertion and Subject-Driven Generationcited by 16
  7. 2022Adma-GAN: Attribute-Driven Memory Augmented GANs for Text-to-Image Generationcited by 6
  8. 2022Text-to-Image Generation via Implicit Visual Guidance and Hypernetworkcited by 4
  9. 2022Learning to Model Multimodal Semantic Alignment for Story Visualizationcited by 3
  10. 2024Data Extrapolation for Text-to-image Generation on Small Datasetscited by 1