# Multilingual Retrieval Augmented Generation for Culturally-Sensitive Tasks: A Benchmark for Cross-lingual Robustness

arXiv: https://arxiv.org/abs/2410.01171 · HTML: https://mlsanity.com/paper/2410.01171/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation.
- Builds on 17 of its 27 references that are in the map.
- First posted on arXiv 2 Oct 2024.

## Placed in

- Retrieval-augmented generation / [Generation and context integration](https://mlsanity.com/rag/generation-and-context-integration/)

## Builds on (most cited first, 17 of 17)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://mlsanity.com/paper/2402.03216/) (2024)
- [Large Language Models Can Be Easily Distracted by Irrelevant Context](https://mlsanity.com/paper/2302.00093/) (2023)
- [TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages](https://mlsanity.com/paper/2003.05002/) (2020)
- [MEGA: Multilingual Evaluation of Generative AI](https://mlsanity.com/paper/2303.12528/) (2023)
- [Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answering](https://mlsanity.com/paper/2307.16877/) (2023)
- [XOR QA: Cross-lingual Open-Retrieval Question Answering](https://mlsanity.com/paper/2010.11856/) (2020)
- [Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Models](https://mlsanity.com/paper/2310.10378/) (2023)
- [One Question Answering Model for Many Languages with Cross-lingual Dense Passage Retrieval](https://mlsanity.com/paper/2107.11976/) (2021)
- [This Land is {Your, My} Land: Evaluating Geopolitical Biases in Language Models](https://mlsanity.com/paper/2305.14610/) (2023)
- [Retrieval-augmented generation in multilingual settings](https://mlsanity.com/paper/2407.01463/) (2024)
- [Neural Approaches to Multilingual Information Retrieval](https://mlsanity.com/paper/2209.01335/) (2022)
- [MIA 2022 Shared Task: Evaluating Cross-lingual Open-Retrieval Question Answering for 16 Diverse Languages](https://mlsanity.com/paper/2207.00758/) (2022)
- [Eliciting Better Multilingual Structured Reasoning from LLMs through Code](https://mlsanity.com/paper/2403.02567/) (2024)
- [Uncovering Differences in Persuasive Language in Russian versus English Wikipedia](https://mlsanity.com/paper/2409.19148/) (2024)

## Built on by (most cited first, 14 of 14)

- [On the Consistency of Multilingual Context Utilization in Retrieval-Augmented Generation](https://mlsanity.com/paper/2504.00597/) (2025)
- [Can LLMs Express Personality Across Cultures? Introducing CulturalPersonas for Evaluating Trait Alignment](https://mlsanity.com/paper/2506.05670/) (2025)
- [Linguistic Nepotism: Trading-off Quality for Language Preference in Multilingual RAG](https://mlsanity.com/paper/2509.13930/) (2025)
- [Improving Multilingual Retrieval-Augmented Language Models through Dialectic Reasoning Argumentations](https://mlsanity.com/paper/2504.04771/) (2025)
- [Enhancing Large Language Models with Neurosymbolic Reasoning for Multilingual Tasks](https://mlsanity.com/paper/2506.02483/) (2025)
- [Cost-Efficient Cross-Lingual Retrieval-Augmented Generation for Low-Resource Languages: A Case Study in Bengali Agricultural Advisory](https://mlsanity.com/paper/2601.02065/) (2026)
- [Language Shapes Mental Health Evaluations in Large Language Models](https://mlsanity.com/paper/2603.06910/) (2026)
- [FLANS at SemEval-2026 Task 7: RAG with Open-Sourced Smaller LLMs for Everyday Knowledge Across Diverse Languages and Cultures](https://mlsanity.com/paper/2603.01910/) (2026)
- [The Language of the Question Selects the Market: Query Language and Exit IP as Separable Factors in Commercial Recommendations from a Generative Search Interface](https://mlsanity.com/paper/2608.30052/) (2026)
- [CORAL: Adaptive Retrieval Loop for Culturally-Aligned Multilingual RAG](https://mlsanity.com/paper/2604.25676/) (2026)
- [Where Privacy Risk Lives in English-Source Multilingual RAG: A Stage-Decomposed Audit Across Five Query Languages](https://mlsanity.com/paper/2608.05163/) (2026)
- [CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systems](https://mlsanity.com/paper/2609.22494/) (2026)
- [Where's Waldo? Query-language Preference under Cross-lingual Knowledge Disparities](https://mlsanity.com/paper/2610.00606/) (2026)
- [GeopoliticalQA: Benchmarking Retrieval and Generation over Intelligence Reports](https://mlsanity.com/paper/2610.12617/) (2026)

## Abstract (arXiv)

The paradigm of retrieval-augmented generated (RAG) helps mitigate hallucinations of large language models (LLMs). However, RAG also introduces biases contained within the retrieved documents. These biases can be amplified in scenarios which are multilingual and culturally-sensitive, such as territorial disputes. We thus introduce BordIRLines, a dataset of territorial disputes paired with retrieved Wikipedia documents, across 49 languages. We evaluate the cross-lingual robustness of this RAG setting by formalizing several modes for multilingual retrieval. Our experiments on several LLMs show that incorporating perspectives from diverse languages can in fact improve robustness; retrieving multilingual documents best improves response consistency and decreases geopolitical bias over RAG with purely in-language documents. We also consider how RAG responses utilize presented documents, finding a much wider variance in the linguistic distribution of response citations, when querying in low-resource languages. Our further analyses investigate the various aspects of a cross-lingual RAG pipeline, from retrieval to document contents. We release our benchmark and code to support continued research towards equitable information access across languages at https://huggingface.co/datasets/borderlines/bordirlines.