# Under-resourced studies of under-resourced languages: lemmatization and POS-tagging with LLM annotators for historical Armenian, Georgian, Greek and Syriac

arXiv: https://arxiv.org/abs/2602.15753 · HTML: https://mlsanity.com/paper/2602.15753/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 20 references that are in the map.
- First posted on arXiv 17 Feb 2026.

## Builds on (most cited first, 5 of 5)

- [Large Language Models as Annotators: Enhancing Generalization of NLP Models at Minimal Cost](https://mlsanity.com/paper/2306.15766/) (2023)
- [Improving Lemmatization of Non-Standard Languages with Joint Learning](https://mlsanity.com/paper/1903.06939/) (2019)
- [Taqyim: Evaluating Arabic NLP Tasks Using ChatGPT Models](https://mlsanity.com/paper/2306.16322/) (2023)
- [Why Moli`ere most likely did write his plays](https://mlsanity.com/paper/2001.01595/) (2020)
- [Corpus and Models for Lemmatisation and POS-tagging of Classical French Theatre](https://mlsanity.com/paper/2005.07505/) (2020)

## Built on by (most cited first, 1 of 1)

- [Mapping Armenian Paris: Extracting and Geocoding Commercial Advertisements from the 20th-Century Diaspora Press](https://mlsanity.com/paper/2608.05911/) (2026)

## Abstract (arXiv)

Low-resource languages pose persistent challenges for Natural Language Processing tasks such as lemmatization and part-of-speech (POS) tagging. This paper investigates the capacity of recent large language models (LLMs), including GPT-4 variants and open-weight Mistral models, to address these tasks in few-shot and zero-shot settings for four historically and linguistically diverse under-resourced languages: Ancient Greek, Classical Armenian, Old Georgian, and Syriac. Using a novel benchmark comprising aligned training and out-of-domain test corpora, we evaluate the performance of foundation models across lemmatization and POS-tagging, and compare them with PIE, a task-specific RNN baseline. Our results demonstrate that LLMs, even without fine-tuning, achieve competitive or superior performance in POS-tagging and lemmatization across most languages in few-shot settings. Significant challenges persist for languages characterized by complex morphology and non-Latin scripts, but we demonstrate that LLMs are a credible and relevant option for initiating linguistic annotation tasks in the absence of data, serving as an effective aid for annotation.