# Evaluating Cross-lingual Knowledge Consistency in Code-Mixed vis-a-vis Indian Languages using IndicKLAR

arXiv: https://arxiv.org/abs/2605.29637 · HTML: https://mlsanity.com/paper/2605.29637/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 31 references that are in the map.
- First posted on arXiv 28 May 2026.

## Builds on (most cited first, 16 of 16)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting](https://mlsanity.com/paper/2305.07004/) (2023)
- [Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Models](https://mlsanity.com/paper/2310.10378/) (2023)
- [Do Multilingual Language Models Think Better in English?](https://mlsanity.com/paper/2308.01223/) (2023)
- [IndicLLMSuite: A Blueprint for Creating Pre-training and Fine-Tuning Datasets for Indian Languages](https://mlsanity.com/paper/2403.06350/) (2024)
- [Lost in Multilinguality: Dissecting Cross-lingual Factual Inconsistency in Transformer Language Models](https://mlsanity.com/paper/2504.04264/) (2025)
- [How Reliable is Multilingual LLM-as-a-Judge?](https://mlsanity.com/paper/2505.12201/) (2025)
- [Evaluating the Diversity, Equity and Inclusion of NLP Technology: A Case Study for Indian Languages](https://mlsanity.com/paper/2205.12676/) (2022)
- [MuBench: Assessment of Multilingual Capabilities of Large Language Models Across 61 Languages](https://mlsanity.com/paper/2506.19468/) (2025)
- [Paramanu: Compact and Competitive Monolingual Language Models for Low-Resource Morphologically Rich Indian Languages](https://mlsanity.com/paper/2401.18034/) (2024)
- [Understanding LLMs' Cross-Lingual Context Retrieval: How Good It Is And Where It Comes From](https://mlsanity.com/paper/2504.10906/) (2025)
- [On the Entity-Level Alignment in Crosslingual Consistency](https://mlsanity.com/paper/2510.10280/) (2025)
- [Found in Translation: Measuring Multilingual LLM Consistency as Simple as Translate then Evaluate](https://mlsanity.com/paper/2505.21999/) (2025)
- [Mind the (Language) Gap: Towards Probing Numerical and Cross-Lingual Limits of LVLMs](https://mlsanity.com/paper/2508.17334/) (2025)
- [Do LLMs Need Inherent Reasoning Before Reinforcement Learning? A Study in Korean Self-Correction](https://mlsanity.com/paper/2601.05459/) (2026)
- [Evaluating Contextually Mediated Factual Recall in Multilingual Large Language Models](https://mlsanity.com/paper/2601.12555/) (2026)

## Abstract (arXiv)

Large language models often exhibit a substantial gap between their performance in English and in lower-resourced languages on equivalent knowledge queries---a cross-lingual consistency issue that remains underexplored for Indian languages and their code-mixed counterparts. To study this gap, we introduce IndicKLAR, an Indic extension of the KLAR-CLC benchmark covering 18 of the 22 scheduled Indian languages. For 11 widely used language pairs, we additionally provide code-mixed variants. Both monolingual and code-mixed inputs verified by native speakers. This three-way alignment enables us to examine how knowledge recall consistency varies across English, code-mixed, and native Indian language inputs. Across nine open-weight models, we find that the accuracy gap between native-language and English inputs can reach ~0.50, while code-mixed inputs substantially reduce this gap, bringing performance within ~0.05 of English without any model-level intervention. Motivated by this finding, we evaluate several prompting strategies that differ in how explicitly language conversion is exposed: a two-stage translate-then-answer setup, a one-stage joint translation-and-answer prompt, and Translate-in-Thought (TinT)---a single-step strategy in which the model internally converts the input and outputs only the final answer. Across the native rightarrow code-mixed rightarrow English performance trajectory, we observe a consistent flip point---the transition from incorrect to correct prediction---between the native and code-mixed settings. Notably, this pattern holds both when the code-mixed representation is explicitly provided as input or when the model is prompted to convert internally using TinT.