# Simile Understanding in Text-to-Image Models: An Evaluation Framework

arXiv: https://arxiv.org/abs/2608.04750 · HTML: https://mlsanity.com/paper/2608.04750/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 50 references that are in the map.
- First posted on arXiv 5 Aug 2026.

## Builds on (most cited first, 23 of 23)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [Qwen-Image Technical Report](https://mlsanity.com/paper/2508.02324/) (2025)
- [GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment](https://mlsanity.com/paper/2310.11513/) (2023)
- [T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generation](https://mlsanity.com/paper/2307.06350/) (2023)
- [Experience Grounds Language](https://mlsanity.com/paper/2004.10151/) (2020)
- [Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change](https://mlsanity.com/paper/1605.09096/) (2016)
- [What Will it Take to Fix Benchmarking in Natural Language Understanding?](https://mlsanity.com/paper/2104.02145/) (2021)
- [MERMAID: Metaphor Generation with Symbolism and Discriminative Decoding](https://mlsanity.com/paper/2103.06779/) (2021)
- [T2I-ReasonBench: Benchmarking Reasoning-Informed Text-to-Image Generation](https://mlsanity.com/paper/2508.17472/) (2025)
- [Neural Metaphor Detection in Context](https://mlsanity.com/paper/1808.09653/) (2018)
- [I Spy a Metaphor: Large Language Models and Diffusion Models Co-Create Visual Metaphors](https://mlsanity.com/paper/2305.14724/) (2023)
- [Diffusion Lens: Interpreting Text Encoders in Text-to-Image Pipelines](https://mlsanity.com/paper/2403.05846/) (2024)
- [Metaphor Understanding Challenge Dataset for LLMs](https://mlsanity.com/paper/2403.11810/) (2024)
- [IRFL: Image Recognition of Figurative Language](https://mlsanity.com/paper/2303.15445/) (2023)
- [Understanding Figurative Meaning through Explainable Visual Entailment](https://mlsanity.com/paper/2405.01474/) (2024)
- [Can Pre-trained Language Models Interpret Similes as Smart as Human?](https://mlsanity.com/paper/2203.08452/) (2022)
- [Large Language Model Displays Emergent Ability to Interpret Novel Literary Metaphors](https://mlsanity.com/paper/2308.01497/) (2023)
- [Metaphor and Large Language Models: When Surface Features Matter More than Deep Understanding](https://mlsanity.com/paper/2507.15357/) (2025)
- [TextTIGER: Text-based Intelligent Generation with Entity Prompt Refinement for Text-to-Image Generation](https://mlsanity.com/paper/2504.18269/) (2025)

## Abstract (arXiv)

Similes provide a compact and expressive way to describe visual characteristics in text prompts. Recent text-to-image models (t2i models) can produce visually compelling outputs from simile prompts, yet even frontier models frequently misinterpret the metaphorical vehicle and confuse it with the object. These systematic failures reveal a gap between figurative language and object-level visual grounding in t2i models. To investigate this issue, we propose a scalable evaluation framework for simile understanding. Our framework includes (1) a controlled simile dataset in which metaphorical vehicles are drawn from a predefined set of object-detectable categories and combined with diverse templates, (2) automatic grounding metrics based on YOLO (You Only Look Once) detection, and (3) text encoder layer analysis using Diffusion Lens to track how metaphorical vehicles emerge during generation. Experiments across architecturally diverse t2i models reveal consistent literalization failure patterns. We further discuss potential mitigation strategies for improving simile grounding in t2i models.