Indic-TunedLens: Interpreting Multilingual Models in Indian Languages
Not on the map. Why?
Abstract (arXiv)
Multilingual large language models (LLMs) are increasingly deployed in linguistically diverse regions like India, yet most interpretability tools remain tailored to English. Prior work reveals that LLMs often operate in English centric representation spaces, making cross lingual interpretability a pressing concern. We introduce Indic-TunedLens, a novel interpretability framework specifically for Indian languages that learns shared affine transformations. Unlike the standard Logit Lens, which directly decodes intermediate activations, Indic-TunedLens adjusts hidden states for each target language, aligning them with the target output distributions to enable more faithful decoding of model representations. We evaluate our framework on 10 Indian languages using the MMLU benchmark and find that it significantly improves over SOTA interpretability methods, especially for morphologically rich, low resource languages. Our results provide crucial insights into the layer-wise semantic encoding of multilingual transformers. Our model is available at https://huggingface.co/spaces/MihirRajeshPanchal/IndicTunedLens. Our code is available at https://github.com/MihirRajeshPanchal/IndicTunedLens.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 11 of its 16 references that are in the map. First posted on arXiv 29 Jan 2026.
Builds on
11 of 16 references.
References · 11
- 2017Attention Is All You Needcited by 59,932
- 2022Locating and Editing Factual Associations in GPTcited by 2,609
- 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
- 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 417
- 2023Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedbackcited by 189
- 2025Do Multilingual LLMs Think In English?cited by 75
- 2025Reinforcement Learning from Human Feedbackcited by 53
- 2023Hyperpolyglot LLMs: Cross-Lingual Interpretability in Token Embeddingscited by 19
- 2025Causal Language Control in Multilingual Transformers via Sparse Feature Steeringcited by 12
- 2025RomanLens: The Role Of Latent Romanization In Multilinguality In LLMscited by 11
- 2024Probing self-attention in self-supervised speech models for cross-linguistic differencescited by 3
Built on by
Citing papers per year: 2026: 2.
2 papers.