mlsanityData to 14 Oct 2026

Assessing Agentic Large Language Models in Multilingual National Bias

arXiv 2502.17945 ↗First posted 25 Feb 2025cs.CLcited by 0

On the map: LLM agents and tool use → Social simulation and multi-agent debate

Abstract (arXiv)

Large Language Models have garnered significant attention for their capabilities in multilingual natural language processing, while studies on risks associated with cross biases are limited to immediate context preferences. Cross-language disparities in reasoning-based recommendations remain largely unexplored, with a lack of even descriptive analysis. This study is the first to address this gap. We test LLM's applicability and capability in providing personalized advice across three key scenarios: university applications, travel, and relocation. We investigate multilingual bias in state-of-the-art LLMs by analyzing their responses to decision-making tasks across multiple languages. We quantify bias in model-generated scores and assess the impact of demographic factors and reasoning strategies (e.g., Chain-of-Thought prompting) on bias patterns. Our findings reveal that local language bias is prevalent across different tasks, with GPT-4 and Sonnet reducing bias for English-speaking countries compared to GPT-3.5 but failing to achieve robust multilingual alignment, highlighting broader implications for multilingual AI agents and applications such as education. footnoteCode available at: https://github.com/yiyunya/assessagenticnationalbias

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 20 of its 25 references that are in the map. First posted on arXiv 25 Feb 2025.

Builds on

20 of 25 references.

References · 20

  1. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 701
  2. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  3. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 530
  4. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 437
  5. 2023Having Beer after Prayer? Measuring Cultural Bias in Large Language Modelscited by 153
  6. 2021Systematic Inequalities in Language Technology Performance across the World's Languagescited by 120
  7. 2024Language Model Alignment in Multilingual Trolley Problemscited by 46
  8. 2024Evaluating Cultural Adaptability of a Large Language Model via Simulation of Synthetic Personascited by 37
  9. 2024Gender Bias in Large Language Models across Multiple Languagescited by 35
  10. 2024MBBQ: A Dataset for Cross-Lingual Comparison of Stereotypes in Generative LLMscited by 35
  11. 2023This Land is {Your, My} Land: Evaluating Geopolitical Biases in Language Modelscited by 32
  12. 2023Investigating Subtler Biases in LLMs: Ageism, Beauty, Institutional, and Nationality Bias in Generative Modelscited by 32
8 more
  1. 2023On Evaluating and Mitigating Gender Biases in Multilingual Settingscited by 31
  2. 2024The Silicon Ceiling: Auditing GPT's Race and Gender Biases in Hiringcited by 29
  3. 2023Global Voices, Local Biases: Socio-Cultural Prejudices across Languagescited by 23
  4. 2023Language Representation Projection: Can We Transfer Factual Knowledge across Languages in Multilingual Language Models?cited by 19
  5. 2024Quite Good, but Not Enough: Nationality Bias in Large Language Models -- A Case Study of ChatGPTcited by 13
  6. 2024Do Multilingual Large Language Models Mitigate Stereotype Bias?cited by 13
  7. 2024What an Elegant Bridge: Multilingual LLMs are Biased Similarly in Different Languagescited by 5
  8. 2023Dissecting Bias of ChatGPT in College Major Recommendationscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Mitigating Cultural Bias in LLMs via Multi-Agent Cultural Debate
  2. 2025Could Thinking Multilingually Empower LLM Reasoning?
  3. 2024Interpreting Bias in Large Language Models: A Feature-Based Approach
  4. 2025Ticket-Bench: A Kickoff for Multilingual and Regionalized Agent Evaluation
  5. 2025Do Multilingual LLMs Think In English?
  6. 2026CogBias: Measuring and Mitigating Cognitive Bias in Large Language Models
  7. 2025Language-Specific Latent Process Hinders Cross-Lingual Performance
  8. 2025Think Globally, Group Locally: Evaluating LLMs Using Multi-Lingual Word Grouping Games