# Script Choice in LLMs: Evidence for Late-Layer Commitment

arXiv: https://arxiv.org/abs/2609.28784 · HTML: https://mlsanity.com/paper/2609.28784/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 19 references that are in the map.
- First posted on arXiv 23 Sep 2026.

## Builds on (most cited first, 12 of 12)

- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [BERT Rediscovers the Classical NLP Pipeline](https://mlsanity.com/paper/1905.05950/) (2019)
- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [Designing and Interpreting Probes with Control Tasks](https://mlsanity.com/paper/1909.03368/) (2019)
- [Dissecting Contextual Word Embeddings: Architecture and Representation](https://mlsanity.com/paper/1808.08949/) (2018)
- [Not All Layers of LLMs Are Necessary During Inference](https://mlsanity.com/paper/2403.02181/) (2024)
- [Exploring Multilingual Probing in Large Language Models: A Cross-Language Analysis](https://mlsanity.com/paper/2409.14459/) (2024)
- [RomanLens: The Role Of Latent Romanization In Multilinguality In LLMs](https://mlsanity.com/paper/2502.07424/) (2025)
- [Extracting Paragraphs from LLM Token Activations](https://mlsanity.com/paper/2409.06328/) (2024)
- [LinguaMap: Which Layers of LLMs Speak Your Language and How to Tune Them?](https://mlsanity.com/paper/2601.20009/) (2026)
- [LLM Layers Immediately Correct Each Other](https://mlsanity.com/paper/2609.07876/) (2026)
- [ParaScopes: What do Language Models Activations Encode About Future Text?](https://mlsanity.com/paper/2511.00180/) (2025)

## Built on by (most cited first, 1 of 1)

- [Are Language Models Script-Aware?](https://mlsanity.com/paper/2610.08037/) (2026)

## Abstract (arXiv)

In this paper, we investigate how script knowledge is distributed across the layers of LLMs using two complementary interpretability methods: logistic regression probing and logit-lens analysis. Our probing experiments reveal a clear asymmetry: both the input script and the instructed output script are encoded in the earliest layers of the network, while, in contrast, commitment to the actual output script emerges only in the final layers, with the model's intermediate representations defaulting to Latin throughout most of the layers. This two-stage process is confirmed by logit-lens analyses, which show that script commitment consistently occurs at the very last layers of the LLMs. Together with the weaker script-following performance observed in smaller models, these results form a converging body of evidence linking script commitment to model depth, with broader implications for the design of sufficiently deep, inclusive multilingual architectures.