mlsanityData to 7 Oct 2026

MiQA: A Benchmark for Inference on Metaphorical Questions

arXiv 2210.07993 ↗First posted 14 Oct 2022cs.CLcited by 9

Not on the map. Why?

Abstract (arXiv)

We propose a benchmark to assess the capability of large language models to reason with conventional metaphors. Our benchmark combines the previously isolated topics of metaphor detection and commonsense reasoning into a single task that requires a model to make inferences by accurately selecting between the literal and metaphorical register. We examine the performance of state-of-the-art pre-trained models on binary-choice tasks and find a large discrepancy between the performance of small and very large models, going from chance to near-human level. We also analyse the largest model in a generative setting and find that although human performance is approached, careful multiple-shot prompting is required.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 11 of its 31 references that are in the map. First posted on arXiv 14 Oct 2022.

Builds on

11 of 31 references.

References · 11

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  3. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  4. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  5. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  6. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  7. 2019Measuring Compositional Generalization: A Comprehensive Method on Realistic Datacited by 280
  8. 2021Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Basescited by 107
  9. 2022Testing the Ability of Language Models to Interpret Figurative Languagecited by 67
  10. 2021MelBERT: Metaphor Detection via Contextualized Late Interaction using Metaphorical Identification Theoriescited by 46
  11. 2021It's not Rocket Science : Interpreting Figurative Language in Narrativescited by 39

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 3; 2025: 5.

9 papers.

  1. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  2. 2024Metaphor Understanding Challenge Dataset for LLMscited by 30
  3. 2025Metaphor and Large Language Models: When Surface Features Matter More than Deep Understandingcited by 8
  4. 2024Meta4XNLI: A Crosslingual Parallel Corpus for Metaphor Detection and Interpretationcited by 5
  5. 2025KRISTEVA: Close Reading as a Novel Task for Benchmarking Interpretive Reasoningcited by 4
  6. 2025Conceptual Metaphor Theory as a Prompting Paradigm for Large Language Modelscited by 3
  7. 2025Hummus: A Dataset of Humorous Multimodal Metaphor Usecited by 2
  8. 2025Idiom Understanding as a Tool to Measure the Dialect Gapcited by 2
  9. 2024Chinese Metaphor Recognition Using a Multi-stage Prompting Large Language Modelcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A Dual-Perspective Metaphor Detection Framework Using Large Language Models
  2. 2026Metaphors are a Source of Cross-Domain Misalignment of Large Reasoning Models
  3. 2022Psychologically-informed chain-of-thought prompts for metaphor understanding in large language models
  4. 2025Enhancing multimodal analogical reasoning with Logic Augmented Generation
  5. 2026Do Vision-Language Models See or Guess? Measuring and Reducing Textual-Prior Reliance with a Phrasing-Controlled Benchmark
  6. 2026BrainBench: Exposing the Commonsense Reasoning Gap in Large Language Models
  7. 2025Metaphor identification using large language models: A comparison of RAG, prompt engineering, and fine-tuning
  8. 2024Science is Exploration: Computational Frontiers for Conceptual Metaphor Theory