Scale-Dependent Semantic Dynamics Revealed by Allan Deviation
Not on the map. Why?
Abstract (arXiv)
While language progresses through a sequence of semantic states, the underlying dynamics of this progression remain elusive. Here, we treat the semantic progression of written text as a stochastic trajectory in a high-dimensional state space. We utilize Allan deviation, a tool from precision metrology, to analyze the stability of meaning by treating ordered sentence embeddings as a displacement signal. Our analysis reveals two distinct dynamical regimes: short-time power-law scaling, which differentiates creative literature from technical texts, and a long-time crossover to a stability-limited noise floor. We find that while large language models successfully mimic the local scaling statistics of human text, they exhibit a systematic reduction in their stability horizon. These results establish semantic coherence as a measurable physical property, offering a framework to differentiate the nuanced dynamics of human cognition from the patterns generated by algorithmic models.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 6 of its 21 references that are in the map. First posted on arXiv 29 Jan 2026.
Builds on
6 of 21 references.
References · 6
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2020Longformer: The Long-Document Transformercited by 3,413
- 2024Know When To Stop: A Study of Semantic Drift in Text Generationcited by 18
- 2017Do Neural Nets Learn Statistical Laws behind Natural Language?cited by 16
Built on by
Not cited yet.