FAIRGAMER: Evaluating Social Biases in LLM-Based Video Game NPCs
Not on the map. Why?
Abstract (arXiv)
Large Language Models (LLMs) have increasingly enhanced or replaced traditional Non-Player Characters (NPCs) in video games. However, these LLM-based NPCs inherit underlying social biases (e.g., race or class), posing fairness risks during in-game interactions. To address the limited exploration of this issue, we introduce FairGamer, the first benchmark to evaluate social biases across three interaction patterns: transaction, cooperation, and competition. FairGamer assesses four bias types, including class, race, age, and nationality, across 12 distinct evaluation tasks using a novel metric, FairMCV. Our evaluation of seven frontier LLMs reveals that: (1) models exhibit biased decision-making, with Grok-4-Fast demonstrating the highest bias (average FairMCV = 76.9%); and (2) larger LLMs display more severe social biases, suggesting that increased model capacity inadvertently amplifies these biases. We release FairGamer at https://github.com/Anonymous999-xxx/FairGamer to facilitate future research on NPC fairness.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 29 of its 56 references that are in the map. First posted on arXiv 25 Aug 2025.
Builds on
29 of 56 references.
References · 29
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023Large Language Modelscited by 780
- 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 488
- 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 412
- 2019On Measuring Social Biases in Sentence Encoderscited by 398
- 2023Having Beer after Prayer? Measuring Cultural Bias in Large Language Modelscited by 153
- 2024Large Language Models are Inconsistent and Biased Evaluatorscited by 153
- 2023SmartPlay: A Benchmark for LLMs as Intelligent Agentscited by 125
- 2024GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluationscited by 120
- 2023Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendationcited by 114
- 2024Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinementcited by 114
- 2024Systematic Biases in LLM Simulations of Debatescited by 99
17 more
- 2024Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Eracited by 98
- 2023Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiationcited by 96
- 2023Can Large Language Models Serve as Rational Players in Game Theory? A Systematic Analysiscited by 92
- 2023ChatGPT or Grammarly? Evaluating ChatGPT on Grammatical Error Correction Benchmarkcited by 77
- 2023MAgIC: Investigation of Large Language Model Powered Multi-Agent in Cognition, Adaptability, Rationality and Collaborationcited by 57
- 2024Towards Implicit Bias Detection and Mitigation in Multi-Agent LLM Interactionscited by 51
- 2024LLM economicus? Mapping the Behavioral Biases of LLMs via Utility Theorycited by 47
- 2023GameGPT: Multi-agent Collaborative Framework for Game Developmentcited by 43
- 2023Fake Alignment: Are LLMs Really Aligned Well?cited by 39
- 2023GameEval: Evaluating LLMs on Conversational Gamescited by 37
- 2024Decoding Biases: Automated Methods and LLM Judges for Gender Bias Detection in Language Modelscited by 33
- 2023FFT: Towards Harmlessness Evaluation and Analysis for LLMs with Factuality, Fairness, Toxicitycited by 25
- 2024Player-Driven Emergence in LLM-Driven Game Narrativecited by 16
- 2025AI Sees Your Location, But With A Bias Toward The Wealthy Worldcited by 14
- 2025Where Fact Ends and Fairness Begins: Redefining AI Bias Evaluation through Cognitive Biasescited by 6
- 2024Game Plot Design with an LLM-powered Assistant: An Empirical Study with Game Designerscited by 4
- 2023General Phrase Debiaser: Debiasing Masked Language Models at a Multi-Token Levelcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.