mlsanityData to 7 Oct 2026

LANGSAE EDITING: Improving Multilingual Information Retrieval via Post-hoc Language Identity Removal

arXiv 2601.04768 ↗First posted 8 Jan 2026cs.CL cs.IRcited by 0

Not on the map. Why?

Abstract (arXiv)

Dense retrieval in multilingual settings often searches over mixed-language collections, yet multilingual embeddings encode language identity alongside semantics. This language signal can inflate similarity for same-language pairs and crowd out relevant evidence written in other languages. We propose LANGSAE EDITING, a post-hoc sparse autoencoder trained on pooled embeddings that enables controllable removal of language-identity signal directly in vector space. The method identifies language-associated latent units using cross-language activation statistics, suppresses these units at inference time, and reconstructs embeddings in the original dimensionality, making it compatible with existing vector databases without retraining the base encoder or re-encoding raw text. Experiments across multiple languages show consistent improvements in ranking quality and cross-language coverage, with especially strong gains for script-distinct languages.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 35 references that are in the map. First posted on arXiv 8 Jan 2026.

Builds on

27 of 35 references.

References · 27

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  5. 2017Billion-scale similarity search with GPUscited by 2,477
  6. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  7. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  8. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 853
  9. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  10. 2020Language-agnostic BERT Sentence Embeddingcited by 621
  11. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 575
  12. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 300
15 more
  1. 2013k-Sparse Autoencoderscited by 209
  2. 2017All-but-the-Top: Simple and Effective Postprocessing for Word Representationscited by 205
  3. 2021Mr. TyDi: A Multi-lingual Benchmark for Dense Retrievalcited by 118
  4. 2021TSDAE: Using Transformer-based Sequential Denoising Auto-Encoder for Unsupervised Sentence Embedding Learningcited by 116
  5. 2021mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Datasetcited by 116
  6. 2020On the Language Neutrality of Pre-trained Multilingual Representationscited by 82
  7. 2021WhiteningBERT: An Easy Unsupervised Sentence Embedding Approachcited by 68
  8. 2019How Language-Neutral is Multilingual BERT?cited by 51
  9. 2020Inducing Language-Agnostic Multilingual Representationscited by 48
  10. 2024Discovering Low-rank Subspaces for Language-agnostic Multilingual Representationscited by 18
  11. 2021A Simple and Effective Method To Eliminate the Self Language Bias in Multilingual Representationscited by 16
  12. 2023Soft Prompt Decoding for Multilingual Dense Retrievalcited by 10
  13. 2022Language Agnostic Multilingual Information Retrieval with Contrastive Learningcited by 8
  14. 2022Unsupervised Context Aware Sentence Representation Pretraining for Multi-lingual Dense Retrievalcited by 5
  15. 2025Language Bias in Information Retrieval: The Nature of the Beast and Mitigation Methodscited by 2

Built on by

Not cited yet.