# Debiasing Multilingual LLMs in Cross-lingual Latent Space

arXiv: https://arxiv.org/abs/2508.17948 · HTML: https://mlsanity.com/paper/2508.17948/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 47 references that are in the map.
- First posted on arXiv 25 Aug 2025.

## Builds on (most cited first, 30 of 32)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models](https://mlsanity.com/paper/2010.00133/) (2020)
- [Word Translation Without Parallel Data](https://mlsanity.com/paper/1710.04087/) (2017)
- [Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection](https://mlsanity.com/paper/2004.07667/) (2020)
- [Unified Concept Editing in Diffusion Models](https://mlsanity.com/paper/2308.14761/) (2023)
- [Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP](https://mlsanity.com/paper/2103.00453/) (2021)
- [Domain Generalization for Object Recognition with Multi-task Autoencoders](https://mlsanity.com/paper/1508.07680/) (2015)
- [Measuring and Reducing Gendered Correlations in Pre-trained Models](https://mlsanity.com/paper/2010.06032/) (2020)
- ["Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters](https://mlsanity.com/paper/2310.09219/) (2023)
- [Towards Debiasing Sentence Representations](https://mlsanity.com/paper/2007.08100/) (2020)
- [Aya Expanse: Combining Research Breakthroughs for a New Multilingual Frontier](https://mlsanity.com/paper/2412.04261/) (2024)
- [When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?](https://mlsanity.com/paper/1804.06323/) (2018)
- [Debiasing Pre-trained Contextualised Embeddings](https://mlsanity.com/paper/2101.09523/) (2021)
- [Mitigating Language-Dependent Ethnic Bias in BERT](https://mlsanity.com/paper/2109.05704/) (2021)
- [HumanEval-XL: A Multilingual Code Generation Benchmark for Cross-lingual Natural Language Generalization](https://mlsanity.com/paper/2402.16694/) (2024)
- [Gender-preserving Debiasing for Pre-trained Word Embeddings](https://mlsanity.com/paper/1906.00742/) (2019)
- [Explicit Alignment Objectives for Multilingual Bidirectional Encoders](https://mlsanity.com/paper/2010.07972/) (2020)
- [Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment](https://mlsanity.com/paper/2106.06381/) (2021)
- [Multilingual BERT Post-Pretraining Alignment](https://mlsanity.com/paper/2010.12547/) (2020)
- [ERNIE-Code: Beyond English-Centric Cross-lingual Pretraining for Programming Languages](https://mlsanity.com/paper/2212.06742/) (2022)
- [PreAlign: Boosting Cross-Lingual Transfer by Early Establishment of Multilingual Alignment](https://mlsanity.com/paper/2407.16222/) (2024)
- [Investigating Bias in Multilingual Language Models: Cross-Lingual Transfer of Debiasing Techniques](https://mlsanity.com/paper/2310.10310/) (2023)
- [Concept Space Alignment in Multilingual LLMs](https://mlsanity.com/paper/2410.01079/) (2024)
- [SemEval-2025 Task 7: Multilingual and Crosslingual Fact-Checked Claim Retrieval](https://mlsanity.com/paper/2505.10740/) (2025)
- [Aurora-M: Open Source Continual Pre-training for Multilingual Language and Code](https://mlsanity.com/paper/2404.00399/) (2024)

## Built on by (most cited first, 2 of 2)

- [Latent-Space Intervention for Cross-Lingual Factual Consistency: Consistency Improvements without Accuracy Drops](https://mlsanity.com/paper/2608.28860/) (2026)
- [Analyzing and Mitigating Cross-Lingual Degradation in Multilingual Medical VQA](https://mlsanity.com/paper/2608.22363/) (2026)

## Abstract (arXiv)

Debiasing techniques such as SentDebias aim to reduce bias in large language models (LLMs). Previous studies have evaluated their cross-lingual transferability by directly applying these methods to LLM representations, revealing their limited effectiveness across languages. In this work, we therefore propose to perform debiasing in a joint latent space rather than directly on LLM representations. We construct a well-aligned cross-lingual latent space using an autoencoder trained on parallel TED talk scripts. Our experiments with Aya-expanse and two debiasing techniques across four languages (English, French, German, Dutch) demonstrate that a) autoencoders effectively construct a well-aligned cross-lingual latent space, and b) applying debiasing techniques in the learned cross-lingual latent space significantly improves both the overall debiasing performance and cross-lingual transferability.