# Metaphor identification using large language models: A comparison of RAG, prompt engineering, and fine-tuning

arXiv: https://arxiv.org/abs/2509.24866 · HTML: https://mlsanity.com/paper/2509.24866/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 56 references that are in the map.
- First posted on arXiv 29 Sep 2025.

## Builds on (most cited first, 13 of 13)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Is GPT-3 a Good Data Annotator?](https://mlsanity.com/paper/2212.10450/) (2022)
- [AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators](https://mlsanity.com/paper/2303.16854/) (2023)
- [A Survey on Long Text Modeling with Transformers](https://mlsanity.com/paper/2302.14502/) (2023)
- [Identifying and Extracting Rare Disease Phenotypes with Large Language Models](https://mlsanity.com/paper/2306.12656/) (2023)
- [Assessing the potential of LLM-assisted annotation for corpus-based pragmatics and discourse analysis: The case of apology](https://mlsanity.com/paper/2305.08339/) (2023)
- [Large corpora and large language models: a replicable method for automating grammatical annotation](https://mlsanity.com/paper/2411.11260/) (2024)
- [Science is Exploration: Computational Frontiers for Conceptual Metaphor Theory](https://mlsanity.com/paper/2410.08991/) (2024)
- [Can GPT-4 learn to analyse moves in research article abstracts?](https://mlsanity.com/paper/2407.15612/) (2024)

## Built on by (most cited first, 3 of 3)

- [LinguistAgent Technical Report: A Reflective Multi-Model Platform for Automated Linguistic Annotation](https://mlsanity.com/paper/2602.05493/) (2026)
- [A Reproducible Multi-Architecture Baseline for Token-Level Chinese Metaphor Identification under the MIPVU Framework](https://mlsanity.com/paper/2605.07170/) (2026)
- [Cross-Dataset Stability of Expert-Informed Skill Prompting and Fine-Tuning for Chinese Metaphor Identification](https://mlsanity.com/paper/2608.25579/) (2026)

## Abstract (arXiv)

Metaphor is a pervasive feature of discourse and a powerful lens for examining cognition, emotion, and ideology. Large-scale analysis, however, has been constrained by the need for manual annotation due to the context-sensitive nature of metaphor. This study investigates the potential of large language models (LLMs) to automate metaphor identification in full texts. We compare three methods: (i) retrieval-augmented generation (RAG), where the model is provided with a codebook and instructed to annotate texts based on its rules and examples; (ii) prompt engineering, where we design task-specific verbal instructions; and (iii) fine-tuning, where the model is trained on hand-coded texts to optimize performance. Within prompt engineering, we test zero-shot, few-shot, and chain-of-thought strategies. Our results show that state-of-the-art closed-source LLMs can achieve high accuracy, with fine-tuning yielding a median F1 score of 0.79. A comparison of human and LLM outputs reveals that most discrepancies are systematic, reflecting well-known grey areas and conceptual challenges in metaphor theory. We propose that LLMs can be used to at least partly automate metaphor identification and can serve as a testbed for developing and refining metaphor identification protocols and the theory that underpins them.