mlsanityData to 7 Oct 2026

Text-to-Image Generation Grounded by Fine-Grained User Attention

arXiv 2011.03775 ↗First posted 7 Nov 2020cs.CV cs.AIcited by 36

Not on the map. Why?

Abstract (arXiv)

Localized Narratives is a dataset with detailed natural language descriptions of images paired with mouse traces that provide a sparse, fine-grained visual grounding for phrases. We propose TReCS, a sequential model that exploits this grounding to generate images. TReCS uses descriptions to retrieve segmentation masks and predict object labels aligned with mouse traces. These alignments are used to select and position masks to generate a fully covered segmentation canvas; the final image is produced by a segmentation-to-image generator using this canvas. This multi-step, retrieval-based approach outperforms existing direct text-to-image generation models on both automatic metrics and human evaluations: overall, its generated images are more photo-realistic and better match descriptions.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 9; 2022: 10; 2023: 9; 2024: 3; 2025: 5.

36 papers.

Sort:

By area:

  1. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  2. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  3. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  4. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  5. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  6. 2022ReCo: Region-Controlled Text-to-Image Generationcited by 177
  7. 2023SneakyPrompt: Jailbreaking Text-to-image Generative Modelscited by 175
  8. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  9. 2023Collaborative Diffusion for Multi-Modal Face Generation and Editingcited by 116
  10. 2024Unveiling and Mitigating Memorization in Text-to-image Diffusion Models through Cross Attentioncited by 57
  11. 2021Text to Image Generation with Semantic-Spatial Aware GANcited by 53
  12. 2021Adversarial Text-to-Image Synthesis: A Reviewcited by 52
24 more
  1. 2021Multimodal Image Synthesis and Editing: The Generative AI Eracited by 52
  2. 2021Integrating Visuospatial, Linguistic and Commonsense Structure into Story Visualizationcited by 51
  3. 2023RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Modelcited by 47
  4. 2022StyleT2I: Toward Compositional and High-Fidelity Text-to-Image Synthesiscited by 33
  5. 2022Simple and Effective Synthesis of Indoor 3D Scenescited by 33
  6. 2023ECLIPSE: A Resource-Efficient Text-to-Image Prior for Image Generationscited by 24
  7. 2021CRD-CGAN: Category-Consistent and Relativistic Constraints for Diverse Text-to-Image Generationcited by 10
  8. 2023ConceptBed: Evaluating Concept Learning Abilities of Text-to-Image Diffusion Modelscited by 10
  9. 2023NarrativePlay: Interactive Narrative Understandingcited by 10
  10. 2022Multimodal Lecture Presentations Dataset: Understanding Multimodality in Educational Slidescited by 9
  11. 2022Markup-to-Image Diffusion Models with Scheduled Samplingcited by 6
  12. 2022IC3D: Image-Conditioned 3D Diffusion for Shape Generationcited by 5
  13. 2022DT2I: Dense Text-to-Image Generation from Region Descriptionscited by 4
  14. 2022Text-to-Image Generation via Implicit Visual Guidance and Hypernetworkcited by 4
  15. 2025MedicalNarratives: Connecting Medical Vision and Language with Localized Narrativescited by 2
  16. 2023The Right Losses for the Right Gains: Improving the Semantic Consistency of Deep Text-to-Image Generation with Distribution-Sensitive Lossescited by 1
  17. 2023Emage: Non-Autoregressive Text-to-Image Generationcited by 1
  18. 2024Offline Evaluation of Set-Based Text-to-Image Generationcited by 1
  19. 2025Replace in Translation: Boost Concept Alignment in Counterfactual Text-to-Imagecited by 1
  20. 2023Style Generation: Image Synthesis based on Coarsely Matched Textscited by 0
  21. 2024AvatarMMC: 3D Head Avatar Generation and Editing with Multi-Modal Conditioningcited by 0
  22. 2025PartStickers: Generating Parts of Objects for Rapid Prototypingcited by 0
  23. 2025TokenProber: Jailbreaking Text-to-image Models via Fine-grained Word Impact Analysiscited by 0
  24. 2025Double Helix Diffusion for Cross-Domain Anomaly Image Generationcited by 0