mlsanityData to 7 Oct 2026

Testing the Ability of Language Models to Interpret Figurative Language

arXiv 2204.12632 ↗First posted 26 Apr 2022cs.CL cs.AIcited by 67

Not on the map. Why?

Abstract (arXiv)

Figurative and metaphorical language are commonplace in discourse, and figurative expressions play an important role in communication and cognition. However, figurative language has been a relatively under-studied area in NLP, and it remains an open question to what extent modern language models can interpret nonliteral phrases. To address this question, we introduce Fig-QA, a Winograd-style nonliteral language understanding task consisting of correctly interpreting paired figurative phrases with divergent meanings. We evaluate the performance of several state-of-the-art language models on this task, and find that although language models achieve performance significantly over chance, they still fall short of human performance, particularly in zero- or few-shot settings. This suggests that further work is needed to improve the nonliteral reasoning capabilities of language models.

Key facts. Cited by 67 papers in the map (data through 7 Oct 2026). Builds on 13 of its 56 references that are in the map. First posted on arXiv 26 Apr 2022.

Builds on

13 of 56 references.

References · 13

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 798
  5. 2018Annotation Artifacts in Natural Language Inference Datacited by 656
  6. 2020Experience Grounds Languagecited by 241
  7. 2021RiddleSense: Reasoning about Riddle Questions Featuring Linguistic Creativity and Commonsense Knowledgecited by 60
  8. 2021It's not Rocket Science : Interpreting Figurative Language in Narrativescited by 39
  9. 2018Neural Metaphor Detection in Contextcited by 34
  10. 2021Back to Square One: Artifact Detection, Training and Commonsense Disentanglement in the Winograd Schemacited by 32
  11. 2022Metaphors in Pre-Trained Language Models: Probing and Generalization Across Datasets and Languagescited by 30
  12. 2021Figurative Language in Recognizing Textual Entailmentcited by 26
1 more
  1. 2021Investigating Robustness of Dialog Models to Popular Figurative Language Constructscited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 11; 2023: 13; 2024: 14; 2025: 21; 2026: 8.

67 papers.

Sort:

By area:

  1. 2022Leveraging Large Language Models for Multiple Choice Question Answeringcited by 199
  2. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  3. 2024Stable LM 2 1.6B Technical Reportcited by 86
  4. 2022FLUTE: Figurative Language Understanding through Textual Explanationscited by 78
  5. 2022A fine-grained comparison of pragmatic language understanding in humans and language modelscited by 69
  6. 2024PUB: A Pragmatics Understanding Benchmark for Assessing LLMs' Pragmatics Capabilitiescited by 58
  7. 2023Multi-lingual and Multi-cultural Figurative Language Understandingcited by 47
  8. 2023Are Multilingual LLMs Culturally-Diverse Reasoners? An Investigation into Multicultural Proverbs and Sayingscited by 32
  9. 2023I Spy a Metaphor: Large Language Models and Diffusion Models Co-Create Visual Metaphorscited by 31
  10. 2024Metaphor Understanding Challenge Dataset for LLMscited by 30
  11. 2025From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Modelscited by 26
  12. 2025Fann or Flop: A Multigenre, Multiera Benchmark for Arabic Poetry Understanding in LLMscited by 15
38 more
  1. 2023Large Language Model Displays Emergent Ability to Interpret Novel Literary Metaphorscited by 12
  2. 2024A Hard Nut to Crack: Idiom Detection with Conversational Large Language Modelscited by 11
  3. 2024mCSQA: Multilingual Commonsense Reasoning Dataset with Unified Creation Strategy by Language Models and Humanscited by 10
  4. 2025Position: Text Embeddings Should Capture Implicit Semantics, Not Just Surface Meaningcited by 10
  5. 2022MiQA: A Benchmark for Inference on Metaphorical Questionscited by 9
  6. 2024Semantics of Multiword Expressions in Transformer-Based Models: A Surveycited by 9
  7. 2025Non-literal Understanding of Number Words by Language Modelscited by 9
  8. 2025Metaphor and Large Language Models: When Surface Features Matter More than Deep Understandingcited by 8
  9. 2022A Survey in Automatic Irony Processing: Linguistic, Cognitive, and Multi-X Perspectivescited by 6
  10. 2023Connecting the Dots in News Analysis: Bridging the Cross-Disciplinary Disparities in Media Bias and Framingcited by 6
  11. 2024Toward Informal Language Processing: Knowledge of Slang in Large Language Modelscited by 6
  12. 2025CoMet: Metaphor-Driven Covert Communication for Multi-Agent Language Gamescited by 6
  13. 2025Are Large Language Models Chronically Online Surfers? A Dataset for Chinese Internet Meme Explanationcited by 6
  14. 2022Leveraging a New Spanish Corpus for Multilingual and Crosslingual Metaphor Detectioncited by 5
  15. 2022EUREKA: EUphemism Recognition Enhanced through Knn-based methods and Augmentationcited by 5
  16. 2024Meta4XNLI: A Crosslingual Parallel Corpus for Metaphor Detection and Interpretationcited by 5
  17. 2025Probing Semantic Alignment, Lexical Invariance, and Syntactic Influence in LLM Metaphor Processingcited by 5
  18. 2024Semantic Change Characterization with LLMs using Rhetoricscited by 4
  19. 2025When People are Floods: Analyzing Dehumanizing Metaphors in Immigration Discourse with Large Language Modelscited by 4
  20. 2025KRISTEVA: Close Reading as a Novel Task for Benchmarking Interpretive Reasoningcited by 4
  21. 2022Detecting Euphemisms with Literal Descriptions and Visual Imagerycited by 3
  22. 2022Harnessing Knowledge and Reasoning for Human-Like Natural Language Generation: A Brief Reviewcited by 3
  23. 2023LMs stand their Ground: Investigating the Effect of Embodiment in Figurative Language Interpretation by Language Modelscited by 3
  24. 2023ViPE: Visualise Pretty-much Everythingcited by 3
  25. 2023Divergences between Language Models and Human Brainscited by 3
  26. 2024PARADISE: Evaluating Implicit Planning Skills of Language Models with Procedural Warnings and Tips Datasetcited by 3
  27. 2025Towards Multimodal Metaphor Understanding: A Chinese Dataset and Model for Metaphor Mapping Identificationcited by 3
  28. 2022Effective Cross-Task Transfer Learning for Explainable Natural Language Inference with T5cited by 2
  29. 2022MAPS-KB: A Million-scale Probabilistic Simile Knowledge Basecited by 2
  30. 2023Does the "most sinfully decadent cake ever" taste good? Answering Yes/No Questions from Figurative Contextscited by 2
  31. 2024Probing Language Models' Gesture Understanding for Enhanced Human-AI Interactioncited by 2
  32. 2025Hummus: A Dataset of Humorous Multimodal Metaphor Usecited by 2
  33. 2025Lost in Variation? Evaluating NLI Performance in Basque and Spanish Geographical Variantscited by 2
  34. 2026I Came, I Saw, I Explained: Benchmarking Multimodal LLMs on Figurative Meaning in Memescited by 2
  35. 2026Not all ANIMALs are equal: metaphorical framing through source domains and semantic framescited by 2
  36. 2023Evaluating Large Language Models' Ability Using a Psychiatric Screening Tool Based on Metaphor and Sarcasm Scenarioscited by 1
  37. 2023Generating Continuations in Multilingual Idiomatic Contextscited by 1
  38. 2024Enabling Natural Zero-Shot Prompting on Encoder Models via Statement-Tuningcited by 1

50 most cited of 67 All 67 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Reasoning Beyond Literal: Cross-style Multimodal Reasoning for Figurative Language Understanding
  2. 2024Understanding Figurative Meaning through Explainable Visual Entailment
  3. 2024Science is Exploration: Computational Frontiers for Conceptual Metaphor Theory
  4. 2024A Dual-Perspective Metaphor Detection Framework Using Large Language Models
  5. 2025Metaphor identification using large language models: A comparison of RAG, prompt engineering, and fine-tuning
  6. 2022Psychologically-informed chain-of-thought prompts for metaphor understanding in large language models
  7. 2024Comparative Study of Multilingual Idioms and Similes in Large Language Models
  8. 2025BengaliFig: A Low-Resource Challenge for Figurative and Culturally Grounded Reasoning in Bengali