mlsanityData to 14 Oct 2026

Semantic Novelty Trajectories in 80,000 Books: A Cross-Corpus Embedding Analysis

arXiv 2603.01791 ↗First posted 2 Mar 2026cs.CL cs.IRcited by 0

Not on the map. Why?

Abstract (arXiv)

I apply Schmidhuber's compression progress theory of interestingness at corpus scale, analyzing semantic novelty trajectories in more than 80,000 books spanning two centuries of English-language publishing. Using sentence-transformer paragraph embeddings and a running-centroid novelty measure, I compare 28,730 pre-1920 Project Gutenberg books (PG19) against 52,796 modern English books (Books3, approximately 1990-2010). The principal findings are fourfold. First, mean paragraph-level novelty is roughly 10% higher in modern books (0.503 vs. 0.459). Second, trajectory circuitousness -- the ratio of cumulative path length to net displacement in embedding space -- nearly doubles in the modern corpus (+67%). Third, convergent narrative curves, in which novelty declines toward a settled semantic register, are 2.3x more common in pre-1920 literature. Fourth, novelty is orthogonal to reader quality ratings (r = -0.002), suggesting that interestingness in Schmidhuber's sense is structurally independent of perceived literary merit. Clustering paragraph-level trajectories via PAA-16 representations reveals eight distinct narrative-shape archetypes whose distribution shifts substantially between eras. All analysis code and an interactive exploration toolkit are publicly available at https://bigfivekiller.online/novelty_hub.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 7 of its 27 references that are in the map. First posted on arXiv 2 Mar 2026.

Builds on

7 of 27 references.

References · 7

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  5. 2019Compressive Transformers for Long-Range Sequence Modellingcited by 726
  6. 2022MTEB: Massive Text Embedding Benchmarkcited by 640
  7. 2016The emotional arcs of stories are dominated by six basic shapescited by 71

Built on by

Not cited yet.