mlsanityData to 7 Oct 2026

Evaluating LLMs for Quotation Attribution in Literary Texts: A Case Study of LLaMa3

arXiv 2406.11380 ↗First posted 17 Jun 2024cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have shown promising results in a variety of literary tasks, often using complex memorized details of narration and fictional characters. In this work, we evaluate the ability of Llama-3 at attributing utterances of direct-speech to their speaker in novels. The LLM shows impressive results on a corpus of 28 novels, surpassing published results with ChatGPT and encoder-based baselines by a large margin. We then validate these results by assessing the impact of book memorization and annotation contamination. We found that these types of memorization do not explain the large performance gain, making Llama-3 the new state-of-the-art for quotation attribution in English literature. We release publicly our code and data.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 13 of its 34 references that are in the map. First posted on arXiv 17 Jun 2024.

Builds on

13 of 34 references.

References · 13

  1. 2020Extracting Training Data from Large Language Modelscited by 2,144
  2. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  3. 2022Data Contamination: From Memorization to Exploitationcited by 234
  4. 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attackscited by 168
  5. 2019BERT for Coreference Resolution: Baselines and Analysiscited by 125
  6. 2023Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4cited by 123
  7. 2022Heroes, Villains, and Victims, and GPT-3: Automated Extraction of Character Roles Without Training Datacited by 27
  8. 2023Personality Understanding of Fictional Characters during Book Readingcited by 20
  9. 2020Measuring Information Propagation in Literary Social Networkscited by 15
  10. 2022The Project Dialogism Novel Corpus: A Dataset for Quotation Attribution in Literary Textscited by 13
  11. 2023Grounding Characters and Places in Narrative Textscited by 9
  12. 2023Improving Automatic Quotation Attribution in Literary Novelscited by 9
1 more
  1. 2023SIG: Speaker Identification in Literature via Prompt-Based Generationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 2; 2026: 4.

8 papers.

  1. 2024Understanding Literary Texts by LLMs: A Case Study of Ancient Chinese Poetrycited by 4
  2. 2024Analyzing Nobel Prize Literature with Large Language Modelscited by 4
  3. 2025Are You There God? Lightweight Narrative Annotation of Christian Fiction with LMscited by 1
  4. 2025Computational Narrative Understanding for Expressive Text-to-Speechcited by 1
  5. 2026Text-to-Stage: Spatial Layouts from Long-form Narrativescited by 0
  6. 2026Attribution Bias in Large Language Modelscited by 0
  7. 2026Reinforcing privacy reasoning in LLMs via normative simulacra from fictioncited by 0
  8. 2026Fast and Accurate Quotation Attribution in Literary Textscited by 0