# Reading Without a Reader: Large Language Models Collapse Reading and Writing into a Single Entangled Code

arXiv: https://arxiv.org/abs/2607.24797 · HTML: https://mlsanity.com/paper/2607.24797/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 72 references that are in the map.
- First posted on arXiv 2 Jul 2026.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned](https://mlsanity.com/paper/1905.09418/) (2019)
- [Are Sixteen Heads Really Better than One?](https://mlsanity.com/paper/1905.10650/) (2019)
- [Quantifying Memorization Across Neural Language Models](https://mlsanity.com/paper/2202.07646/) (2022)
- [Eliciting Latent Predictions from Transformers with the Tuned Lens](https://mlsanity.com/paper/2303.08112/) (2023)
- [The Platonic Representation Hypothesis](https://mlsanity.com/paper/2405.07987/) (2024)
- [Using the Output Embedding to Improve Language Models](https://mlsanity.com/paper/1608.05859/) (2016)
- [Experience Grounds Language](https://mlsanity.com/paper/2004.10151/) (2020)
- [Talking About Large Language Models](https://mlsanity.com/paper/2212.03551/) (2022)
- [Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokens](https://mlsanity.com/paper/2401.17377/) (2024)
- [The Debate Over Understanding in AI's Large Language Models](https://mlsanity.com/paper/2210.13966/) (2022)
- [Similarity of Neural Network Models: A Survey of Functional and Representational Measures](https://mlsanity.com/paper/2305.06329/) (2023)
- [Offline bilingual word vectors, orthogonal transformations and the inverted softmax](https://mlsanity.com/paper/1702.03859/) (2017)
- [Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling](https://mlsanity.com/paper/1611.01462/) (2016)
- [Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora](https://mlsanity.com/paper/2504.08165/) (2025)
- [Understanding Token Probability Encoding in Output Embeddings](https://mlsanity.com/paper/2406.01468/) (2024)
- [Palatable Conceptions of Disembodied Being](https://mlsanity.com/paper/2503.16348/) (2025)
- [Weight Tying Biases Token Embeddings Towards the Output Space](https://mlsanity.com/paper/2603.26663/) (2026)

## Abstract (arXiv)

In the literate human brain, reading and writing doubly dissociate: a ventral decoding route (pure alexia) and a fronto-parietal encoding route (pure agraphia), sharing a partial orthographic core. A decoder-only large language model (LLM) drives both from one autoregressive path optimized on text (a cultural invention, not an evolved instinct). We ask how entangled it is, comparing an input-side ``reading code'' WE with an output-side ``writing code'' WU via an index E∈[0,1] (CKA, Procrustes residual, mutual k-NN) calibrated against an independent-init floor and tied ceiling. On GPT-2, OPT and Pythia (14M--1.4B), untied models hold one coupled but sub-ceiling code (E=0.23--0.35, far above floor) on a non-monotonic couple-then-differentiate trajectory, WU drifting ~3.2× farther than WE in every decile. Equally informative is a negative: the matching behavioural test, that comprehension and production fail together rather than dissociate, cannot be run. For minimal pairs the alexia analogue is empty by theorem: greedy production implies a vocabulary-wide argmax, so it wins the pairwise ranking. Differential-damage indices are not scale-identified: heavy-tailed damage makes linear standardizations collapse onto their larger term, and the rank transform fixing this is bounded, so its null saturates. Both scores also contain the target's log-probability, which alone explains most of their variance and manufactures the apparent coupling. We withdraw a coupling statistic, a cross-level bridge and a separation measure. In a model reading and writing off one next-token distribution, no output-side pair isolates either ability: entanglement needing no index to see. By analogy, not homology, this situates LLMs in the space of possible minds.