# Complex systems approach to natural language

arXiv: https://arxiv.org/abs/2401.02772 · HTML: https://mlsanity.com/paper/2401.02772/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 289 references that are in the map.
- First posted on arXiv 5 Jan 2024.

## Builds on (most cited first, 19 of 19)

- [Recent Trends in Deep Learning Based Natural Language Processing](https://mlsanity.com/paper/1708.02709/) (2017)
- [Epidemic processes in complex networks](https://mlsanity.com/paper/1408.2701/) (2014)
- [Statistical laws in linguistics](https://mlsanity.com/paper/1502.03296/) (2015)
- [Zipf's law holds for phrases, not words](https://mlsanity.com/paper/1406.5181/) (2014)
- [In narrative texts punctuation marks obey the same statistics as words](https://mlsanity.com/paper/1604.00834/) (2016)
- [The Matthew effect in empirical data](https://mlsanity.com/paper/1408.5124/) (2014)
- [Linguistic data mining with complex networks: a stylometric-oriented approach](https://mlsanity.com/paper/1808.05439/) (2018)
- [Heaps' law and Heaps functions in tagged texts: Evidences of their linguistic relevance](https://mlsanity.com/paper/2001.02178/) (2020)
- [Universal versus system-specific features of punctuation usage patterns in~major Western~languages](https://mlsanity.com/paper/2212.11182/) (2022)
- [Zipf's law for word frequencies: word forms versus lemmas in long texts](https://mlsanity.com/paper/1407.8322/) (2014)
- [Two halves of a meaningful text are statistically different](https://mlsanity.com/paper/2004.06474/) (2020)
- [Analysis of English free association network reveals mechanisms of efficient solution of Remote Association Tests](https://mlsanity.com/paper/2008.12219/) (2020)
- [Multiscale characteristics of the emerging global cryptocurrency market](https://mlsanity.com/paper/2010.15403/) (2020)
- [Text characterization based on recurrence networks](https://mlsanity.com/paper/2201.06665/) (2022)
- [Power laws and Self-Organized Criticality in Theory and Nature](https://mlsanity.com/paper/1310.5527/) (2013)
- [Modeling the average shortest path length in growth of word-adjacency networks](https://mlsanity.com/paper/1409.4714/) (2014)
- [Emergence of linguistic laws in human voice](https://mlsanity.com/paper/1610.02736/) (2016)
- [Hierarchical organization of H. Eugene Stanley scientific collaboration community in weighted network representation](https://mlsanity.com/paper/1705.06208/) (2017)
- [Universal features of mountain ridge networks on Earth](https://mlsanity.com/paper/1804.03457/) (2018)

## Built on by (most cited first, 12 of 12)

- [LLM-Generated Natural Language Meets Scaling Laws: New Explorations and Data Augmentation Methods](https://mlsanity.com/paper/2407.00322/) (2024)
- [MagiCodec: Simple Masked Gaussian-Injected Codec for High-Fidelity Reconstruction and Generation](https://mlsanity.com/paper/2506.00385/) (2025)
- [Entropy and type-token ratio in gigaword corpora](https://mlsanity.com/paper/2411.10227/) (2024)
- [Statistics of punctuation in experimental literature -- the remarkable case of "Finnegans Wake" by James Joyce](https://mlsanity.com/paper/2409.00483/) (2024)
- [Punctuation patterns in "Finnegans Wake" by James Joyce are largely translation-invariant](https://mlsanity.com/paper/2501.12954/) (2025)
- [Multifractal hopscotch in "Hopscotch" by Julio Cortazar](https://mlsanity.com/paper/2501.12955/) (2025)
- [Quantifying patterns of punctuation in modern Chinese prose](https://mlsanity.com/paper/2503.04449/) (2025)
- [Study of scaling laws in language families](https://mlsanity.com/paper/2504.01681/) (2025)
- [Quadratic Term Correction on Heaps' Law](https://mlsanity.com/paper/2511.14683/) (2025)
- [Computational lexical analysis of Flamenco genres](https://mlsanity.com/paper/2405.05723/) (2024)
- [Average shortest-path length in word-adjacency networks: Chinese versus English](https://mlsanity.com/paper/2601.06361/) (2026)
- [A Zipf-preserving, long-range correlated surrogate for written language and other symbolic sequences](https://mlsanity.com/paper/2603.02213/) (2026)

## Abstract (arXiv)

The review summarizes the main methodological concepts used in studying natural language from the perspective of complexity science and documents their applicability in identifying both universal and system-specific features of language in its written representation. Three main complexity-related research trends in quantitative linguistics are covered. The first part addresses the issue of word frequencies in texts and demonstrates that taking punctuation into consideration restores scaling whose violation in the Zipf's law is often observed for the most frequent words. The second part introduces methods inspired by time series analysis, used in studying various kinds of correlations in written texts. The related time series are generated on the basis of text partition into sentences or into phrases between consecutive punctuation marks. It turns out that these series develop features often found in signals generated by complex systems, like long-range correlations or (multi)fractal structures. Moreover, it appears that the distances between punctuation marks comply with the discrete variant of the Weibull distribution. In the third part, the application of the network formalism to natural language is reviewed, particularly in the context of the so-called word-adjacency networks. Parameters characterizing topology of such networks can be used for classification of texts, for example, from a stylometric perspective. Network approach can also be applied to represent the organization of word associations. Structure of word-association networks turns out to be significantly different from that observed in random networks, revealing genuine properties of language. Finally, punctuation seems to have a significant impact not only on the language's information-carrying ability but also on its key statistical properties, hence it is recommended to consider punctuation marks on a par with words.