mlsanityData to 14 Oct 2026

Large Language Models and Book Summarization: Reading or Remembering, Which Is Better?

arXiv 2603.09981 ↗First posted 9 Feb 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Summarization is a core task in Natural Language Processing (NLP). Recent advances in Large Language Models (LLMs) and the introduction of large context windows reaching millions of tokens make it possible to process entire books in a single prompt. At the same time, for well-known books, LLMs can generate summaries based only on internal knowledge acquired during training. This raises several important questions: How do summaries generated from internal memory compare to those derived from the full text? Does prior knowledge influence summaries even when the model is given the book as input? In this work, we conduct an experimental evaluation of book summarization with state-of-the-art LLMs. We compare summaries of well-known books produced using (i) only the internal knowledge of the model and (ii) the full text of the book. The results show that having the full text provides more detailed summaries in general, but some books have better scores for the internal knowledge summaries. This puts into question the capabilities of models to perform summarization of long texts, as information learned during training can outperform summarization of the full text in some cases.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 25 references that are in the map. First posted on arXiv 9 Feb 2026.

Builds on

19 of 25 references.

References · 19

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  4. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  5. 2020Extracting Training Data from Large Language Modelscited by 2,189
  6. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  7. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 812
  8. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  9. 2019Text Summarization with Pretrained Encoderscited by 556
  10. 2023Benchmarking Large Language Models for News Summarizationcited by 445
  11. 2021Recursively Summarizing Books with Human Feedbackcited by 290
  12. 2024LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuningcited by 136
7 more
  1. 2024Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinementcited by 114
  2. 2024A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methodscited by 102
  3. 2025How much do language models memorize?cited by 61
  4. 2023Text Summarization Using Large Language Models: A Comparative Study of MPT-7b-instruct, Falcon-7b-instruct, and OpenAI Chat-GPT Modelscited by 36
  5. 2024Understanding the Interplay between Parametric and Contextual Knowledge for Large Language Modelscited by 15
  6. 2024Assessing LLMs for Zero-shot Abstractive Summarization Through the Lens of Relevance Paraphrasingcited by 11
  7. 2025Evaluating book summaries from internal knowledge in Large Language Models: a cross-model and semantic consistency approachcited by 1

Built on by

Not cited yet.