# Rethinking the Relationship between the Power Law and Hierarchical Structures

arXiv: https://arxiv.org/abs/2505.04984 · HTML: https://mlsanity.com/paper/2505.04984/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 85 references that are in the map.
- First posted on arXiv 8 May 2025.

## Builds on (most cited first, 14 of 14)

- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Constituency Parsing with a Self-Attentive Encoder](https://mlsanity.com/paper/1805.01052/) (2018)
- [Multilingual Constituency Parsing with Self-Attention and Pre-Training](https://mlsanity.com/paper/1812.11760/) (2018)
- [Physics of Language Models: Part 1, Learning Hierarchical Language Structures](https://mlsanity.com/paper/2305.13673/) (2023)
- [Towards a theory of how the structure of language is acquired by deep neural networks](https://mlsanity.com/paper/2406.00048/) (2024)
- [Criticality in Formal Languages and Statistical Physics](https://mlsanity.com/paper/1606.06737/) (2016)
- [Evaluating Computational Language Models with Scaling Properties of Natural Language](https://mlsanity.com/paper/1906.09379/) (2019)
- [Do Neural Nets Learn Statistical Laws behind Natural Language?](https://mlsanity.com/paper/1707.04848/) (2017)
- [Mutual Information Scaling and Expressive Power of Sequence Models](https://mlsanity.com/paper/1905.04271/) (2019)
- [eRST: A Signaled Graph Theory of Discourse Relations and Organization](https://mlsanity.com/paper/2403.13560/) (2024)
- [Estimating the Entropy of Linguistic Distributions](https://mlsanity.com/paper/2204.01469/) (2022)
- [Autocorrelations Decay in Texts and Applicability Limits of Language Models](https://mlsanity.com/paper/2305.06615/) (2023)
- [Recursive Bayesian Networks: Generalising and Unifying Probabilistic Context-Free Grammars and Dynamic Bayesian Networks](https://mlsanity.com/paper/2111.01853/) (2021)
- [Towards a more flexible Language of Thought: Bayesian grammar updates after each concept exposure](https://mlsanity.com/paper/1805.06924/) (2018)

## Built on by (most cited first, 1 of 1)

- [Writing as a Self-Organized Critical Process](https://mlsanity.com/paper/2610.05466/) (2026)

## Abstract (arXiv)

Statistical analysis of corpora provides an approach to quantitatively investigate natural languages. This approach has revealed that several power laws consistently emerge across different corpora and languages, suggesting universal mechanisms underlying languages. In particular, the power-law decay of correlations has been interpreted as evidence of underlying hierarchical structures in syntax, semantics, and discourse. This perspective has also been extended beyond corpora produced by human adults, including child speech, birdsong, and chimpanzee action sequences. However, the argument supporting this interpretation has not been empirically tested in natural languages. To address this gap, the present study examines the validity of the argument for syntactic structures. Specifically, we test whether the statistical properties of parse trees align with the assumptions in the argument. Using English and Japanese corpora, we analyze the mutual information, deviations from probabilistic context-free grammars (PCFGs), and other properties in natural language parse trees, as well as in the PCFG that approximates these parse trees. Our results indicate that the assumptions do not hold for syntactic structures and that it is difficult to apply the proposed argument not only to sentences by human adults but also to other domains, highlighting the need to reconsider the relationship between the power law and hierarchical structures.