# Seventeenth-Century Spanish American Notary Records for Fine-Tuning Spanish Large Language Models

arXiv: https://arxiv.org/abs/2406.05812 · HTML: https://mlsanity.com/paper/2406.05812/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 31 references that are in the map.
- First posted on arXiv 9 Jun 2024.

## Builds on (most cited first, 4 of 4)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Spanish Pre-trained BERT Model and Evaluation Data](https://mlsanity.com/paper/2308.02976/) (2023)
- [Pushing The Limit of LLM Capacity for Text Classification](https://mlsanity.com/paper/2402.07470/) (2024)

## Abstract (arXiv)

Large language models have gained tremendous popularity in domains such as e-commerce, finance, healthcare, and education. Fine-tuning is a common approach to customize an LLM on a domain-specific dataset for a desired downstream task. In this paper, we present a valuable resource for fine-tuning LLMs developed for the Spanish language to perform a variety of tasks such as classification, masked language modeling, clustering, and others. Our resource is a collection of handwritten notary records from the seventeenth century obtained from the National Archives of Argentina. This collection contains a combination of original images and transcribed text (and metadata) of 160+ pages that were handwritten by two notaries, namely, Estenban Agreda de Vergara and Nicolas de Valdivia y Brisuela nearly 400 years ago. Through empirical evaluation, we demonstrate that our collection can be used to fine-tune Spanish LLMs for tasks such as classification and masked language modeling, and can outperform pre-trained Spanish models and ChatGPT-3.5/ChatGPT-4o. Our resource will be an invaluable resource for historical text analysis and is publicly available on GitHub.