# Latent-Space Intervention for Cross-Lingual Factual Consistency: Consistency Improvements without Accuracy Drops

arXiv: https://arxiv.org/abs/2608.28860 · HTML: https://mlsanity.com/paper/2608.28860/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 40 references that are in the map.
- First posted on arXiv 28 Aug 2026.

## Builds on (most cited first, 25 of 25)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation](https://mlsanity.com/paper/2106.03193/) (2021)
- [Measuring and Improving Consistency in Pretrained Language Models](https://mlsanity.com/paper/2102.01017/) (2021)
- [Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Models](https://mlsanity.com/paper/2102.00894/) (2021)
- [Aya Expanse: Combining Research Breakthroughs for a New Multilingual Frontier](https://mlsanity.com/paper/2412.04261/) (2024)
- [Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Models](https://mlsanity.com/paper/2310.10378/) (2023)
- [X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Models](https://mlsanity.com/paper/2010.06189/) (2020)
- [How Do Large Language Models Acquire Factual Knowledge During Pretraining?](https://mlsanity.com/paper/2406.11813/) (2024)
- [On the Language Neutrality of Pre-trained Multilingual Representations](https://mlsanity.com/paper/2004.05160/) (2020)
- [Lost in Multilinguality: Dissecting Cross-lingual Factual Inconsistency in Transformer Language Models](https://mlsanity.com/paper/2504.04264/) (2025)
- [Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment](https://mlsanity.com/paper/2106.06381/) (2021)
- [ECLeKTic: a Novel Challenge Set for Evaluation of Cross-Lingual Knowledge Transfer](https://mlsanity.com/paper/2502.21228/) (2025)
- [Bridging the Language Gaps in Large Language Models with Inference-Time Cross-Lingual Intervention](https://mlsanity.com/paper/2410.12462/) (2024)
- [Beneath the Surface of Consistency: Exploring Cross-lingual Knowledge Representation Sharing in LLMs](https://mlsanity.com/paper/2408.10646/) (2024)
- [Language-Specific Latent Process Hinders Cross-Lingual Performance](https://mlsanity.com/paper/2505.13141/) (2025)
- [Paths Not Taken: Understanding and Mending the Multilingual Factual Recall Pipeline](https://mlsanity.com/paper/2505.20546/) (2025)
- [Language Representation Projection: Can We Transfer Factual Knowledge across Languages in Multilingual Language Models?](https://mlsanity.com/paper/2311.03788/) (2023)
- [Multilingual Knowledge Editing with Language-Agnostic Factual Neurons](https://mlsanity.com/paper/2406.16416/) (2024)
- [When Less Language is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasoners](https://mlsanity.com/paper/2505.15257/) (2025)
- [Language Models' Factuality Depends on the Language of Inquiry](https://mlsanity.com/paper/2502.17955/) (2025)
- [Tracing Multilingual Factual Knowledge Acquisition in Pretraining](https://mlsanity.com/paper/2505.14824/) (2025)
- [The Transfer Neurons Hypothesis: An Underlying Mechanism for Language Latent Space Transitions in Multilingual LLMs](https://mlsanity.com/paper/2509.17030/) (2025)
- [Language Steering for Multilingual In-Context Learning](https://mlsanity.com/paper/2602.02326/) (2026)
- [Debiasing Multilingual LLMs in Cross-lingual Latent Space](https://mlsanity.com/paper/2508.17948/) (2025)

## Built on by (most cited first, 2 of 2)

- [Fair Fact-Checking: Closing the Cross-Lingual Gap in LLM Factual Judgement with RoSh](https://mlsanity.com/paper/2609.34678/) (2026)
- [Retrieval Disagreement as a Signal: Diagnosing and Steering Multilingual Dense Embedders](https://mlsanity.com/paper/2610.18526/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) often answer the same factual question differently across languages. We study whether cross-lingual latent-space intervention can reduce this inconsistency. We train layer-specific autoencoders on parallel multilingual representations and apply inference-time corrections to factual QA prompts. We find that latent intervention improves geometric alignment between languages, and that this improvement translates into consistent gains in cross-lingual consistency with English across both open-ended and multiple-choice QA formats, without degrading factual accuracy. In open-ended QA, Spearman's rank correlation between English and non-English languages improves substantially, with gains of 0.16 for English-Arabic and 0.20 for English-Russian pairs. In multiple-choice QA, answer agreement with English improves consistently across both KLAR and mParaRel. Ablations show that AE reconstruction yields consistent gains at no accuracy cost, while PCA projection contributes marginally, and mean-shift produces substantially larger consistency gains in open-ended QA at the cost of some accuracy.