mlsanityData to 7 Oct 2026

DALLE-2 is Seeing Double: Flaws in Word-to-Concept Mapping in Text2Image Models

arXiv 2210.10606 ↗First posted 19 Oct 2022cs.CL cs.LGcited by 50

Not on the map. Why?

Abstract (arXiv)

We study the way DALLE-2 maps symbols (words) in the prompt to their references (entities or properties of entities in the generated image). We show that in stark contrast to the way human process language, DALLE-2 does not follow the constraint that each word has a single role in the interpretation, and sometimes re-use the same symbol for different purposes. We collect a set of stimuli that reflect the phenomenon: we show that DALLE-2 depicts both senses of nouns with multiple senses at once; and that a given word can modify the properties of two distinct entities in the image, or can be depicted as one object and also modify the properties of another object, creating a semantic leakage of properties between entities. Taken together, our study highlights the differences between DALLE-2 and human language processing and opens an avenue for future study on the inductive biases of text-to-image models.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Builds on 6 of its 11 references that are in the map. First posted on arXiv 19 Oct 2022.

Builds on

6 of 11 references.

References · 6

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  4. 2022A very preliminary analysis of DALL-E 2cited by 82
  5. 2022Testing Relational Understanding in Text-Guided Image Generationcited by 59
  6. 2022Underspecification in Scene Description-to-Depiction Taskscited by 30

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 15; 2024: 19; 2025: 7; 2026: 5.

50 papers.

Sort:

By area:

  1. 2023Teaching CLIP to Count to Tencited by 194
  2. 2023Linguistic Binding in Diffusion Models: Enhancing Attribute Correspondence through Attention Map Alignmentcited by 169
  3. 2023DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusioncited by 167
  4. 2023What You See is What You Read? Improving Text-Image Alignment Evaluationcited by 105
  5. 2023Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Taskcited by 91
  6. 2023DreamSync: Aligning Text-to-Image Generation with Image Understanding Feedbackcited by 57
  7. 2024Make It Count: Text-to-Image Generation with an Accurate Number of Objectscited by 47
  8. 2023Testing AI on language comprehension tasks reveals insensitivity to underlying meaningcited by 45
  9. 2023The Hidden Language of Diffusion Modelscited by 34
  10. 2024Emergence of Hidden Capabilities: Exploring Learning Dynamics in Concept Spacecited by 33
  11. 2022DALL-E 2 Fails to Reliably Capture Common Syntactic Processescited by 31
  12. 2022When are Lemons Purple? The Concept Association Bias of Vision-Language Modelscited by 23
38 more
  1. 2023Auditing Gender Presentation Differences in Text-to-Image Modelscited by 22
  2. 2023Mismatch Quest: Visual and Textual Feedback for Image-Text Misalignmentcited by 18
  3. 2024Object-Conditioned Energy-Based Attention Map Alignment in Text-to-Image Diffusion Modelscited by 18
  4. 2024DiffusionPID: Interpreting Diffusion via Partial Information Decompositioncited by 16
  5. 2024Language in Vivo vs. in Silico: Size Matters but Larger Language Models Still Do Not Comprehend Language on a Par with Humans Due to Impenetrable Semantic Referencecited by 15
  6. 2024Does Liking Yellow Imply Driving a School Bus? Semantic Leakage in Language Modelscited by 14
  7. 2022Pragmatics in Language Grounding: Phenomena, Tasks, and Modeling Approachescited by 12
  8. 2024Text Guided Image Editing with Automatic Concept Locating and Forgettingcited by 12
  9. 2024Object-Attribute Binding in Text-to-Image Generation: Evaluation and Controlcited by 10
  10. 2025Image Generation from Contextually-Contradictory Promptscited by 9
  11. 2022Schr"odinger's Bat: Diffusion Models Sometimes Generate Polysemous Words in Superpositioncited by 8
  12. 2024Global-Local Image Perceptual Score (GLIPS): Evaluating Photorealistic Quality of AI-Generated Imagescited by 7
  13. 2024Creativity in AI: Progresses and Challengescited by 7
  14. 2024Inverse Painting: Reconstructing The Painting Processcited by 6
  15. 2023Vision Meets Definitions: Unsupervised Visual Word Sense Disambiguation Incorporating Gloss Informationcited by 5
  16. 2023Total Selfie: Generating Full-Body Selfiescited by 5
  17. 2024Swing-by Dynamics in Concept Learning and Compositional Generalizationcited by 5
  18. 2024GRADE: Quantifying Sample Diversity in Text-to-Image Modelscited by 5
  19. 2024EvalGIM: A Library for Evaluating Generative Image Modelscited by 5
  20. 2025Geometrical Properties of Text Token Embeddings for Strong Semantic Binding in Text-to-Image Generationcited by 5
  21. 2023Transferring Visual Attributes from Natural Language to Verified Image Generationcited by 4
  22. 2024Human-Centric Aware UAV Trajectory Planning in Search and Rescue Missions Employing Multi-Objective Reinforcement Learning with AHP and Similarity-Based Experience Replaycited by 4
  23. 2025Follow the Flow: On Information Flow Across Textual Tokens in Text-to-Image Modelscited by 4
  24. 2024DefectTwin: When LLM Meets Digital Twin for Railway Defect Inspectioncited by 3
  25. 2023Affect-Conditioned Image Generationcited by 2
  26. 2025Scaling Down Semantic Leakage: Investigating Associative Bias in Smaller Language Modelscited by 2
  27. 2025DeLeaker: Dynamic Inference-Time Reweighting For Semantic Leakage Mitigation in Text-to-Image Modelscited by 2
  28. 2023The Quo Vadis of the Relationship between Language and Large Language Modelscited by 1
  29. 2024Explicitly Representing Syntax Improves Sentence-to-layout Prediction of Unexpected Situationscited by 1
  30. 2024Classification-Denoising Networkscited by 1
  31. 2025Un-Doubling Diffusion: LLM-guided Disambiguation of Homonym Duplicationcited by 1
  32. 2026GLAD: Generative Language-Assisted Visual Tracking for Low-Semantic Templatescited by 1
  33. 2024TextureMeDefect: LLM-based Defect Texture Generation for Railway Components on Mobile Devicescited by 0
  34. 2025Quantum Visual Word Sense Disambiguation: Unraveling Ambiguities Through Quantum Inference Modelcited by 0
  35. 2026Morphological Addressing of Identity Basins in Text-to-Image Diffusion Modelscited by 0
  36. 2026Training-Free Object-Background Compositional T2I via Dynamic Spatial Guidance and Multi-Path Pruningcited by 0
  37. 2026Anchoring and Steering Diffusion: Enhancing the Faithfulness of Text-to-Image Generation at Inference Timecited by 0
  38. 2026Where did the ambiguity go? Examining how multimodal models interpret polysemous wordscited by 0