# LinguistAgent Technical Report: A Reflective Multi-Model Platform for Automated Linguistic Annotation

arXiv: https://arxiv.org/abs/2602.05493 · HTML: https://mlsanity.com/paper/2602.05493/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 2 of its 17 references that are in the map.
- First posted on arXiv 5 Feb 2026.

## Builds on (most cited first, 2 of 2)

- [Science is Exploration: Computational Frontiers for Conceptual Metaphor Theory](https://mlsanity.com/paper/2410.08991/) (2024)
- [Metaphor identification using large language models: A comparison of RAG, prompt engineering, and fine-tuning](https://mlsanity.com/paper/2509.24866/) (2025)

## Abstract (arXiv)

Data annotation remains a significant bottleneck in the field of humanities and social sciences, particularly for complex linguistic tasks such as metaphor identification. While Large Language Models (LLMs) show promise, a significant gap remains between the theoretical capability of LLMs and their practical utility for researchers. This paper introduces LinguistAgent, an integrated, user-friendly platform that leverages a reflective multi-model architecture to automate linguistic annotation. The platform comprises an Annotator and an optional Reviewer to simulate a peer-review process. This platform supports comparative experiments across three main paradigms: Prompt Engineering (Zero-shot/Few-shot/Chain-of-thought), Retrieval-Augmented Generation, and Fine-tuning. We demonstrate LinguistAgent's efficacy by replicating the task of metaphor identification from a published study, which provides real-time token-level evaluation (F1 and Cohen's kappa) against human gold standards. The application and codes are released on https://github.com/Bingru-Li/LinguistAgent.