# If the Sources Could Talk: Evaluating Large Language Models for Research Assistance in History

arXiv: https://arxiv.org/abs/2310.10808 · HTML: https://mlsanity.com/paper/2310.10808/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 88 references that are in the map.
- First posted on arXiv 16 Oct 2023.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021)
- [Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond](https://mlsanity.com/paper/2304.13712/) (2023)
- [Orca: Progressive Learning from Complex Explanation Traces of GPT-4](https://mlsanity.com/paper/2306.02707/) (2023)
- [Opportunities and Risks of LLMs for Scalable Deliberation with Polis](https://mlsanity.com/paper/2306.11932/) (2023)
- [Neural Language Models for Nineteenth-Century English](https://mlsanity.com/paper/2105.11321/) (2021)

## Built on by (most cited first, 5 of 5)

- [Provocations from the Humanities for Generative AI Research](https://mlsanity.com/paper/2502.19190/) (2025)
- [A Survey of Large Language Models in Discipline-specific Research: Challenges, Methods and Opportunities](https://mlsanity.com/paper/2507.08425/) (2025)
- [Hybrid Intelligence for Digital Humanities](https://mlsanity.com/paper/2406.15374/) (2024)
- [Computational Analysis of Character Development in Holocaust Testimonies](https://mlsanity.com/paper/2412.17063/) (2024)
- [Hey Chat, Can You Teach Me? Structuring Socratic Dialogue for Human Learning in the Wild](https://mlsanity.com/paper/2606.11744/) (2026)

## Abstract (arXiv)

The recent advent of powerful Large-Language Models (LLM) provides a new conversational form of inquiry into historical memory (or, training data, in this case). We show that by augmenting such LLMs with vector embeddings from highly specialized academic sources, a conversational methodology can be made accessible to historians and other researchers in the Humanities. Concretely, we evaluate and demonstrate how LLMs have the ability of assisting researchers while they examine a customized corpora of different types of documents, including, but not exclusive to: (1). primary sources, (2). secondary sources written by experts, and (3). the combination of these two. Compared to established search interfaces for digital catalogues, such as metadata and full-text search, we evaluate the richer conversational style of LLMs on the performance of two main types of tasks: (1). question-answering, and (2). extraction and organization of data. We demonstrate that LLMs semantic retrieval and reasoning abilities on problem-specific tasks can be applied to large textual archives that have not been part of the its training data. Therefore, LLMs can be augmented with sources relevant to specific research projects, and can be queried privately by researchers.