# Linear Script Representations in Speech Foundation Models Enable Zero-Shot Transliteration

arXiv: https://arxiv.org/abs/2601.02906 · HTML: https://mlsanity.com/paper/2601.02906/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 43 references that are in the map.
- First posted on arXiv 6 Jan 2026.

## Builds on (most cited first, 26 of 26)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [Refusal in Language Models Is Mediated by a Single Direction](https://mlsanity.com/paper/2406.11717/) (2024)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets](https://mlsanity.com/paper/2310.06824/) (2023)
- [FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech](https://mlsanity.com/paper/2205.12446/) (2022)
- [Understanding and Mitigating Language Confusion in LLMs](https://mlsanity.com/paper/2406.20052/) (2024)
- [Refusal Direction is Universal Across Safety-Aligned Languages](https://mlsanity.com/paper/2505.17306/) (2025)
- [Processing South Asian Languages Written in the Latin Script: the Dakshina Dataset](https://mlsanity.com/paper/2007.01176/) (2020)
- [Reducing Language confusion for Code-switching Speech Recognition with Token-level Language Diarization](https://mlsanity.com/paper/2210.14567/) (2022)
- [RomanSetu: Efficiently unlocking multilingual capabilities of Large Language Models via Romanization](https://mlsanity.com/paper/2401.14280/) (2024)
- [Refusal in LLMs is an Affine Function](https://mlsanity.com/paper/2411.09003/) (2024)
- [OWSM v4: Improving Open Whisper-Style Speech Models via Data Scaling and Cleaning](https://mlsanity.com/paper/2506.00338/) (2025)
- [Bridging the Language Gaps in Large Language Models with Inference-Time Cross-Lingual Intervention](https://mlsanity.com/paper/2410.12462/) (2024)
- [EmoSteer-TTS: Fine-Grained and Training-Free Emotion-Controllable Text-to-Speech via Activation Steering](https://mlsanity.com/paper/2508.03543/) (2025)
- [Romanization-based Large-scale Adaptation of Multilingual Language Models](https://mlsanity.com/paper/2304.08865/) (2023)
- [Towards Zero-Shot Code-Switched Speech Recognition](https://mlsanity.com/paper/2211.01458/) (2022)
- [POWSM: A Phonetic Open Whisper-Style Speech Foundation Model](https://mlsanity.com/paper/2510.24992/) (2025)
- [RomanLens: The Role Of Latent Romanization In Multilinguality In LLMs](https://mlsanity.com/paper/2502.07424/) (2025)
- [Emergence of Linear Truth Encodings in Language Models](https://mlsanity.com/paper/2510.15804/) (2025)
- [Cross-Lingual Transfer Learning for Speech Translation](https://mlsanity.com/paper/2407.01130/) (2024)
- [SARSteer: Safeguarding Large Audio-Language Models via Safe-Ablated Refusal Steering](https://mlsanity.com/paper/2510.17633/) (2025)
- [Silence is Golden: Mitigating Hallucinations in Large Audio-Language Models via Layer-Weighted Vector Steering](https://mlsanity.com/paper/2510.12851/) (2025)
- [Phonetic and Visual Priors for Decipherment of Informal Romanization](https://mlsanity.com/paper/2005.02517/) (2020)
- [LAMA-UT: Language Agnostic Multilingual ASR through Orthography Unification and Language-Specific Transliteration](https://mlsanity.com/paper/2412.15299/) (2024)
- [Happiness is Sharing a Vocabulary: A Study of Transliteration Methods](https://mlsanity.com/paper/2510.10827/) (2025)

## Abstract (arXiv)

Multilingual speech foundation models such as Whisper are trained on web-scale data, where data for each language consists of a myriad of regional varieties. However, different regional varieties often employ different scripts to write the same language, rendering speech recognition output also subject to non-determinism in the output script. To mitigate this problem, we show that script is linearly encoded in the activation space of multilingual speech models, and that modifying activations at inference time enables direct control over output script. We find the addition of such script vectors to activations at test time can induce script change even in unconventional language-script pairings (e.g. Italian in Cyrillic and Japanese in Latin script). We apply this approach to inducing post-hoc control over the script of speech recognition output, where we observe competitive performance across all model sizes of Whisper.