mlsanityData to 7 Oct 2026

Towards Multimodal Metaphor Understanding: A Chinese Dataset and Model for Metaphor Mapping Identification

arXiv 2501.02434 ↗First posted 5 Jan 2025cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

Metaphors play a crucial role in human communication, yet their comprehension remains a significant challenge for natural language processing (NLP) due to the cognitive complexity involved. According to Conceptual Metaphor Theory (CMT), metaphors map a target domain onto a source domain, and understanding this mapping is essential for grasping the nature of metaphors. While existing NLP research has focused on tasks like metaphor detection and sentiment analysis of metaphorical expressions, there has been limited attention to the intricate process of identifying the mappings between source and target domains. Moreover, non-English multimodal metaphor resources remain largely neglected in the literature, hindering a deeper understanding of the key elements involved in metaphor interpretation. To address this gap, we developed a Chinese multimodal metaphor advertisement dataset (namely CM3D) that includes annotations of specific target and source domains. This dataset aims to foster further research into metaphor comprehension, particularly in non-English languages. Furthermore, we propose a Chain-of-Thought (CoT) Prompting-based Metaphor Mapping Identification Model (CPMMIM), which simulates the human cognitive process for identifying these mappings. Drawing inspiration from CoT reasoning and Bi-Level Optimization (BLO), we treat the task as a hierarchical identification problem, enabling more accurate and interpretable metaphor mapping. Our experimental results demonstrate the effectiveness of CPMMIM, highlighting its potential for advancing metaphor comprehension in NLP. Our dataset and code are both publicly available to encourage further advancements in this field.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 29 of its 69 references that are in the map. First posted on arXiv 5 Jan 2025.

Builds on

29 of 69 references.

References · 29

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2020End-to-End Object Detection with Transformerscited by 7,384
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  8. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  9. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  10. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  11. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  12. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 740
17 more
  1. 2022Using cognitive psychology to understand GPT-3cited by 389
  2. 2022Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detectioncited by 313
  3. 2022Language models show human-like content effects on reasoning taskscited by 171
  4. 2021Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyondcited by 151
  5. 2022Few-shot training LLMs for project-specific code-summarizationcited by 128
  6. 2021CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generationcited by 81
  7. 2023Reasoning Implicit Sentiment with Chain-of-Thought Promptingcited by 79
  8. 2022Testing the Ability of Language Models to Interpret Figurative Languagecited by 67
  9. 2023T-SciQ: Teaching Multimodal Chain-of-Thought Reasoning via Mixed Large Language Model Signals for Science Question Answeringcited by 59
  10. 2021MelBERT: Metaphor Detection via Contextualized Late Interaction using Metaphorical Identification Theoriescited by 46
  11. 2021Prompting Contrastive Explanations for Commonsense Reasoning Taskscited by 45
  12. 2024A Chain-of-Thought Prompting Approach with LLMs for Evaluating Students' Formative Assessment Responses in Sciencecited by 31
  13. 2023Multi-modal Latent Space Learning for Chain-of-Thought Reasoning in Language Modelscited by 24
  14. 2022Psychologically-informed chain-of-thought prompts for metaphor understanding in large language modelscited by 14
  15. 2024Enhancing Semantics in Multimodal Chain of Thought via Soft Negative Samplingcited by 8
  16. 2019Task-Oriented Convex Bilevel Optimization with Latent Feasibilitycited by 6
  17. 2013Mobile Multimedia Recommendation in Smart Communities: A Surveycited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Large Language Models for Subjective Language Understanding: A Surveycited by 9
  2. 2026Towards High-Level Semantic Intelligencecited by 0
  3. 2026M3R-Bench: A Unified Benchmark for Evidence-Grounded Multimodal Metaphor Understandingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024CMDAG: A Chinese Metaphor Dataset with Annotated Grounds as CoT for Boosting Metaphor Generation
  2. 2024Science is Exploration: Computational Frontiers for Conceptual Metaphor Theory
  3. 2025Metaphor identification using large language models: A comparison of RAG, prompt engineering, and fine-tuning
  4. 2024A Dual-Perspective Metaphor Detection Framework Using Large Language Models
  5. 2024Metaphor Understanding Challenge Dataset for LLMs
  6. 2026I Came, I Saw, I Explained: Benchmarking Multimodal LLMs on Figurative Meaning in Memes
  7. 2025Hummus: A Dataset of Humorous Multimodal Metaphor Use
  8. 2025Probing Semantic Alignment, Lexical Invariance, and Syntactic Influence in LLM Metaphor Processing