# MelBERT: Metaphor Detection via Contextualized Late Interaction using Metaphorical Identification Theories

arXiv: https://arxiv.org/abs/2104.13615 · HTML: https://mlsanity.com/paper/2104.13615/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 45 references that are in the map.
- First posted on arXiv 28 Apr 2021.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [Document Expansion by Query Prediction](https://mlsanity.com/paper/1904.08375/) (2019)
- [Neural Metaphor Detection in Context](https://mlsanity.com/paper/1808.09653/) (2018)
- [Grasping the Finer Point: A Supervised Similarity Network for Metaphor Detection](https://mlsanity.com/paper/1709.00575/) (2017)

## Built on by (most cited first, 30 of 46)

- [MemeCap: A Dataset for Captioning and Interpreting Memes](https://mlsanity.com/paper/2305.13703/) (2023)
- [MathBERT: A Pre-trained Language Model for General NLP Tasks in Mathematics Education](https://mlsanity.com/paper/2106.07340/) (2021)
- [Metaphors in Pre-Trained Language Models: Probing and Generalization Across Datasets and Languages](https://mlsanity.com/paper/2203.14139/) (2022)
- [Metaphor Understanding Challenge Dataset for LLMs](https://mlsanity.com/paper/2403.11810/) (2024)
- [MetaCLUE: Towards Comprehensive Visual Metaphors Research](https://mlsanity.com/paper/2212.09898/) (2022)
- [Hierarchical Attention Network for Explainable Depression Detection on Twitter Aided by Metaphor Concept Mappings](https://mlsanity.com/paper/2209.07494/) (2022)
- [FrameBERT: Conceptual Metaphor Detection with Frame Embedding Learning](https://mlsanity.com/paper/2302.04834/) (2023)
- [Metaphor Detection with Effective Context Denoising](https://mlsanity.com/paper/2302.05611/) (2023)
- [Metaphor Detection via Explicit Basic Meanings Modelling](https://mlsanity.com/paper/2305.17268/) (2023)
- [A Match Made in Heaven: A Multi-task Framework for Hyperbole and Metaphor Detection](https://mlsanity.com/paper/2305.17480/) (2023)
- [Cultural Bias Matters: A Cross-Cultural Benchmark Dataset and Sentiment-Enriched Model for Understanding Multimodal Metaphors](https://mlsanity.com/paper/2506.06987/) (2025)
- [MiQA: A Benchmark for Inference on Metaphorical Questions](https://mlsanity.com/paper/2210.07993/) (2022)
- [Adversarial Multi-task Learning for End-to-end Metaphor Detection](https://mlsanity.com/paper/2305.16638/) (2023)
- [Unveiling the Invisible: Captioning Videos with Metaphors](https://mlsanity.com/paper/2406.04886/) (2024)
- [Metaphorical Polysemy Detection: Conventional Metaphor meets Word Sense Disambiguation](https://mlsanity.com/paper/2212.08395/) (2022)
- [Leveraging a New Spanish Corpus for Multilingual and Crosslingual Metaphor Detection](https://mlsanity.com/paper/2210.10358/) (2022)
- [Survey in Characterizing Semantic Change](https://mlsanity.com/paper/2402.19088/) (2024)
- [Meta4XNLI: A Crosslingual Parallel Corpus for Metaphor Detection and Interpretation](https://mlsanity.com/paper/2404.07053/) (2024)
- [Semantic Change Characterization with LLMs using Rhetorics](https://mlsanity.com/paper/2407.16624/) (2024)
- [When People are Floods: Analyzing Dehumanizing Metaphors in Immigration Discourse with Large Language Models](https://mlsanity.com/paper/2502.13246/) (2025)
- [Probing Semantic Alignment, Lexical Invariance, and Syntactic Influence in LLM Metaphor Processing](https://mlsanity.com/paper/2510.04120/) (2025)
- [Multilingual Multi-Figurative Language Detection](https://mlsanity.com/paper/2306.00121/) (2023)
- [Finding Challenging Metaphors that Confuse Pretrained Language Models](https://mlsanity.com/paper/2401.16012/) (2024)
- [Towards Multimodal Metaphor Understanding: A Chinese Dataset and Model for Metaphor Mapping Identification](https://mlsanity.com/paper/2501.02434/) (2025)
- [ContrastWSD: Enhancing Metaphor Detection with Word Sense Disambiguation Following the Metaphor Identification Procedure](https://mlsanity.com/paper/2309.03103/) (2023)
- [An Expectation-Realization Model for Metaphor Detection](https://mlsanity.com/paper/2311.03963/) (2023)
- [A Dual-Perspective Metaphor Detection Framework Using Large Language Models](https://mlsanity.com/paper/2412.17332/) (2024)
- [Interpretable Chinese Metaphor Identification via LLM-Assisted MIPVU Rule Script Generation: A Comparative Protocol Study](https://mlsanity.com/paper/2603.10784/) (2026)
- [Not all ANIMALs are equal: metaphorical framing through source domains and semantic frames](https://mlsanity.com/paper/2604.20454/) (2026)
- [OYXOY: A Modern NLP Test Suite for Modern Greek](https://mlsanity.com/paper/2309.07009/) (2023)

## Abstract (arXiv)

Automated metaphor detection is a challenging task to identify metaphorical expressions of words in a sentence. To tackle this problem, we adopt pre-trained contextualized models, e.g., BERT and RoBERTa. To this end, we propose a novel metaphor detection model, namely metaphor-aware late interaction over BERT (MelBERT). Our model not only leverages contextualized word representation but also benefits from linguistic metaphor identification theories to distinguish between the contextual and literal meaning of words. Our empirical results demonstrate that MelBERT outperforms several strong baselines on four benchmark datasets, i.e., VUA-18, VUA-20, MOH-X, and TroFi.