mlsanityData to 14 Oct 2026

Metaphors in Pre-Trained Language Models: Probing and Generalization Across Datasets and Languages

arXiv 2203.14139 ↗First posted 26 Mar 2022cs.CLcited by 30

Not on the map. Why?

Abstract (arXiv)

Human languages are full of metaphorical expressions. Metaphors help people understand the world by connecting new concepts and domains to more familiar ones. Large pre-trained language models (PLMs) are therefore assumed to encode metaphorical knowledge useful for NLP systems. In this paper, we investigate this hypothesis for PLMs, by probing metaphoricity information in their encodings, and by measuring the cross-lingual and cross-dataset generalization of this information. We present studies in multiple metaphor detection datasets and in four languages (i.e., English, Spanish, Russian, and Farsi). Our extensive experiments suggest that contextual representations in PLMs do encode metaphorical knowledge, and mostly in their middle layers. The knowledge is transferable between languages and datasets, especially when the annotation is consistent across training and testing sets. Our findings give helpful insights for both cognitive and NLP scientists.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Builds on 26 of its 52 references that are in the map. First posted on arXiv 26 Mar 2022.

Builds on

26 of 52 references.

References · 26

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  5. 2018Deep contextualized word representationscited by 2,290
  6. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
  7. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
  8. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
  9. 2019How multilingual is Multilingual BERT?cited by 645
  10. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 457
  11. 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 438
  12. 2018Learning Models with Uniform Performance via Distributionally Robust Optimizationcited by 335
14 more
  1. 2020Pretrained Transformers Improve Out-of-Distribution Robustnesscited by 294
  2. 2020Information-Theoretic Probing with Minimum Description Lengthcited by 232
  3. 2020Deep Learning Based Text Classification: A Comprehensive Reviewcited by 141
  4. 2021Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Surveycited by 130
  5. 2021Pretrained Language Models for Text Generation: A Surveycited by 95
  6. 2021MelBERT: Metaphor Detection via Contextualized Late Interaction using Metaphorical Identification Theoriescited by 46
  7. 2020What does it mean to be language-agnostic? Probing multilingual sentence encoders for typological propertiescited by 23
  8. 2020Machine Reading Comprehension: The Role of Contextualized Language Models and Beyondcited by 21
  9. 2020Quantifying the Contextualization of Word Representations with Semantic Class Probingcited by 18
  10. 2021Not All Models Localize Linguistic Knowledge in the Same Place: A Layer-wise Probing on BERToids' Representationscited by 16
  11. 2019Probing for Semantic Classes: Diagnosing the Meaning Content of Word Embeddingscited by 15
  12. 2019Probing Multilingual Sentence Representations With X-Probecited by 13
  13. 2016Intrinsic Subspace Evaluation of Word Embedding Representationscited by 5
  14. 2019LINSPECTOR WEB: A Multilingual Probing Suite for Word Representationscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 9; 2024: 7; 2025: 7; 2026: 5.

30 papers.

Sort:

By area:

  1. 2023Having Beer after Prayer? Measuring Cultural Bias in Large Language Modelscited by 153
  2. 2022Testing the Ability of Language Models to Interpret Figurative Languagecited by 67
  3. 2023Multi-lingual and Multi-cultural Figurative Language Understandingcited by 47
  4. 2023MemeCap: A Dataset for Captioning and Interpreting Memescited by 46
  5. 2024Holmes: A Benchmark to Assess the Linguistic Competence of Language Modelscited by 26
  6. 2023IRFL: Image Recognition of Figurative Languagecited by 23
  7. 2025Representation Engineering for Large-Language Models: Survey and Research Challengescited by 21
  8. 2023A Match Made in Heaven: A Multi-task Framework for Hyperbole and Metaphor Detectioncited by 10
  9. 2024MMTE: Corpus and Metrics for Evaluating Machine Translation Quality of Metaphorical Languagecited by 10
  10. 2025Cultural Bias Matters: A Cross-Cultural Benchmark Dataset and Sentiment-Enriched Model for Understanding Multimodal Metaphorscited by 10
  11. 2025Metaphor and Large Language Models: When Surface Features Matter More than Deep Understandingcited by 8
  12. 2024Unveiling the Invisible: Captioning Videos with Metaphorscited by 7
18 more
  1. 2025Linguistic Interpretability of Transformer-based Language Models: a systematic reviewcited by 7
  2. 2024Dive into the Chasm: Probing the Gap between In- and Cross-Topic Generalizationcited by 6
  3. 2024Toward Informal Language Processing: Knowledge of Slang in Large Language Modelscited by 6
  4. 2024Meta4XNLI: A Crosslingual Parallel Corpus for Metaphor Detection and Interpretationcited by 5
  5. 2025When People are Floods: Analyzing Dehumanizing Metaphors in Immigration Discourse with Large Language Modelscited by 5
  6. 2025Probing Semantic Alignment, Lexical Invariance, and Syntactic Influence in LLM Metaphor Processingcited by 5
  7. 2024Science is Exploration: Computational Frontiers for Conceptual Metaphor Theorycited by 4
  8. 2022Probing with Noise: Unpicking the Warp and Weft of Embeddingscited by 3
  9. 2023Multilingual Multi-Figurative Language Detectioncited by 3
  10. 2023Evaluating Large Language Models' Ability Using a Psychiatric Screening Tool Based on Metaphor and Sarcasm Scenarioscited by 2
  11. 2023Estimating Semantic Similarity between In-Domain and Out-of-Domain Samplescited by 1
  12. 2026Surprisal and Metaphor Novelty Judgments: Moderate Correlations and Divergent Scaling Effects Revealed by Corpus-Based and Synthetic Datasetscited by 1
  13. 2023Systematic word meta-sense extensioncited by 0
  14. 2025Mechanistic Interpretability of Socio-Political Frames in Language Modelscited by 0
  15. 2026Post-Hoc Understanding of Metaphor Processing in Decoder-Only Language Models via Conditional Scale Entropycited by 0
  16. 2026Towards High-Level Semantic Intelligencecited by 0
  17. 2026Figurative Justice: Detecting metaphors in Hindi judgements with qualitative assessment and transformerscited by 0
  18. 2026Cross-Dataset Stability of Expert-Informed Skill Prompting and Fine-Tuning for Chinese Metaphor Identificationcited by 0