# A Tale of Two Structures: Do LLMs Capture the Fractal Complexity of Language?

arXiv: https://arxiv.org/abs/2502.14924 · HTML: https://mlsanity.com/paper/2502.14924/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Feb 2025.

## Built on by (most cited first, 3 of 3)

- [FIRE-Bench: Evaluating AI Agents on the Rediscovery of Scientific Insights](https://mlsanity.com/paper/2602.02905/) (2026)
- [Critical Percolation as a Synthetic Data Model for Interpretability](https://mlsanity.com/paper/2606.20347/) (2026)
- [Learning Fractional-Order Dynamics from a Single Trajectory](https://mlsanity.com/paper/2609.18127/) (2026)

## Abstract (arXiv)

Language exhibits a fractal structure in its information-theoretic complexity (i.e. bits per token), with self-similarity across scales and long-range dependence (LRD). In this work, we investigate whether large language models (LLMs) can replicate such fractal characteristics and identify conditions-such as temperature setting and prompting method-under which they may fail. Moreover, we find that the fractal parameters observed in natural language are contained within a narrow range, whereas those of LLMs' output vary widely, suggesting that fractal parameters might prove helpful in detecting a non-trivial portion of LLM-generated texts. Notably, these findings, and many others reported in this work, are robust to the choice of the architecture; e.g. Gemini 1.0 Pro, Mistral-7B and Gemma-2B. We also release a dataset comprising of over 240,000 articles generated by various LLMs (both pretrained and instruction-tuned) with different decoding temperatures and prompting methods, along with their corresponding human-generated texts. We hope that this work highlights the complex interplay between fractal properties, prompting, and statistical mimicry in LLMs, offering insights for generating, evaluating and detecting synthetic texts.