# Emergence of order in random languages

arXiv: https://arxiv.org/abs/1902.07516 · HTML: https://mlsanity.com/paper/1902.07516/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Feb 2019.

## Built on by (most cited first, 5 of 5)

- [Towards a theory of how the structure of language is acquired by deep neural networks](https://mlsanity.com/paper/2406.00048/) (2024)
- [How transformers learn structured data: insights from hierarchical filtering](https://mlsanity.com/paper/2408.15138/) (2024)
- [Integrating Randomness in Large Language Models: A Linear Congruential Generator Approach for Generating Clinically Relevant Content](https://mlsanity.com/paper/2407.03582/) (2024)
- [Robustness of the Random Language Model](https://mlsanity.com/paper/2309.14913/) (2023)
- [Strahler Number of Natural Language Sentences in Comparison with Random Trees](https://mlsanity.com/paper/2307.02697/) (2023)

## Abstract (arXiv)

We consider languages generated by weighted context-free grammars. It is shown that the behaviour of large texts is controlled by saddle-point equations for an appropriate generating function. We then consider ensembles of grammars, in particular the Random Language Model of E. DeGiuli, Phys. Rev. Lett., 122, 128301, 2019. This model is solved in the replica-symmetric ansatz, which is valid in the high-temperature, disordered phase. It is shown that in the phase in which languages carry information, the replica symmetry must be broken.