Reading Without a Reader: Large Language Models Collapse Reading and Writing into a Single Entangled Code
Not on the map. Why?
Abstract (arXiv)
In the literate human brain, reading and writing doubly dissociate: a ventral decoding route (pure alexia) and a fronto-parietal encoding route (pure agraphia), sharing a partial orthographic core. A decoder-only large language model (LLM) drives both from one autoregressive path optimized on text (a cultural invention, not an evolved instinct). We ask how entangled it is, comparing an input-side ``reading code'' WE with an output-side ``writing code'' WU via an index E∈[0,1] (CKA, Procrustes residual, mutual k-NN) calibrated against an independent-init floor and tied ceiling. On GPT-2, OPT and Pythia (14M--1.4B), untied models hold one coupled but sub-ceiling code (E=0.23--0.35, far above floor) on a non-monotonic couple-then-differentiate trajectory, WU drifting ~3.2× farther than WE in every decile. Equally informative is a negative: the matching behavioural test, that comprehension and production fail together rather than dissociate, cannot be run. For minimal pairs the alexia analogue is empty by theorem: greedy production implies a vocabulary-wide argmax, so it wins the pairwise ranking. Differential-damage indices are not scale-identified: heavy-tailed damage makes linear standardizations collapse onto their larger term, and the rank transform fixing this is bounded, so its null saturates. Both scores also contain the target's log-probability, which alone explains most of their variance and manufactures the apparent coupling. We withdraw a coupling statistic, a cross-level bridge and a separation measure. In a model reading and writing off one next-token distribution, no output-side pair isolates either ability: entanglement needing no index to see. By analogy, not homology, this situates LLMs in the space of possible minds.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 72 references that are in the map. First posted on arXiv 2 Jul 2026.
Builds on
31 of 72 references.
References · 31
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2017Graph Attention Networkscited by 8,923
- 2020Scaling Laws for Neural Language Modelscited by 6,586
- 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
- 2022Training Compute-Optimal Large Language Modelscited by 3,019
- 2020Extracting Training Data from Large Language Modelscited by 2,189
- 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
- 2019Similarity of Neural Network Representations Revisitedcited by 1,755
19 more
- 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 1,003
- 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 856
- 2019Are Sixteen Heads Really Better than One?cited by 816
- 2022Quantifying Memorization Across Neural Language Modelscited by 763
- 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
- 2024The Platonic Representation Hypothesiscited by 382
- 2016Using the Output Embedding to Improve Language Modelscited by 272
- 2020Experience Grounds Languagecited by 242
- 2022Talking About Large Language Modelscited by 175
- 2024Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokenscited by 153
- 2022The Debate Over Understanding in AI's Large Language Modelscited by 142
- 2023Similarity of Neural Network Models: A Survey of Functional and Representational Measurescited by 131
- 2017Offline bilingual word vectors, orthogonal transformations and the inverted softmaxcited by 96
- 2016Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modelingcited by 92
- 2025Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corporacited by 88
- 2024Understanding Token Probability Encoding in Output Embeddingscited by 6
- 2025Palatable Conceptions of Disembodied Beingcited by 5
- 2026Weight Tying Biases Token Embeddings Towards the Output Spacecited by 3
- 2025Leveraging Human Production-Interpretation Asymmetries to Test LLM Cognitive Plausibilitycited by 2
Built on by
Not cited yet.