mlsanityData to 14 Oct 2026

AI Alignment through a Game-theoretic Lens: A Survey

arXiv 2608.27910 ↗First posted 28 Aug 2026cs.AI cs.CL cs.GTcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

As large language models and increasingly capable AI agents are deployed in high-risk settings, aligning them with complex human values has become a central challenge. Existing alignment methods, while effective in improving helpfulness, harmlessness, and controllability, often struggle to capture real-world preferences that are context-dependent, non-transitive, and shaped by dynamic multi-party interactions. This survey reviews AI alignment through a game-theoretic lens. Specifically, it organizes recent progress around key game-theoretic elements and synthesizes the literature along three challenges: preference diversity, alignment priority, and temporal dynamics. This perspective clarifies where current alignment methods genuinely benefit from game-theoretic analysis, where the framework is looser, and what challenges remain in building robust, adaptive, and verifiable AI systems.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 91 of its 143 references that are in the map. First posted on arXiv 28 Aug 2026.

Builds on

91 of 143 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Intrinsic Barriers and Practical Pathways for Human-AI Alignment: An Agreement-Based Complexity Analysis9 Feb
  2. 2024Representative Social Choice: From Learning Theory to AI Alignment31 Oct
  3. 2024Position: Towards Bidirectional Human-AI Alignment13 Jun
  4. 2024Collective Constitutional AI: Aligning a Language Model with Public Input12 Jun
  5. 2024AI Alignment with Changing and Influenceable Reward Functions28 May
  6. 2024Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedback16 Apr
  7. 2024Agent Alignment in Evolving Social Norms9 Jan
  8. 2023AI Alignment and Social Choice: Fundamental Limitations and Policy Implications24 Oct
  9. 2023Training Socially Aligned Language Models on Simulated Social Interactions26 May

Other references · 82

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  4. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  5. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  6. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  7. 2016Concrete Problems in AI Safetycited by 1,642
  8. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  9. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,395
  10. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  11. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 1,011
  12. 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 912
38 more
  1. 2016Inherent Trade-Offs in the Fair Determination of Risk Scorescited by 586
  2. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 557
  3. 2024GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Modelscited by 550
  4. 2020Artificial Intelligence, Values and Alignmentcited by 448
  5. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 367
  6. 2022The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Modelscited by 328
  7. 2019On Tiny Episodic Memories in Continual Learningcited by 307
  8. 2024Self-Play Preference Optimization for Language Model Alignmentcited by 232
  9. 2020Understanding the Role of Training Regimes in Continual Learningcited by 180
  10. 2024Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferencescited by 178
  11. 2023RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Modelscited by 175
  12. 2024Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewardscited by 148
  13. 2020Adversarial Continual Learningcited by 142
  14. 2024MaxMin-RLHF: Alignment with Diverse Human Preferencescited by 123
  15. 2024Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsalcited by 118
  16. 2024Secret Collusion among AI Agents: Multi-Agent Deception via Steganographycited by 113
  17. 2024Revisiting Catastrophic Forgetting in Large Language Model Tuningcited by 99
  18. 2023Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiationcited by 96
  19. 2024Group Robust Preference Optimization in Reward-free RLHFcited by 95
  20. 2023Can Large Language Models Serve as Rational Players in Game Theory? A Systematic Analysiscited by 92
  21. 2024Human Alignment of Large Language Models through Online Preference Optimisationcited by 91
  22. 2024Self-playing Adversarial Language Game Enhances LLM Reasoningcited by 83
  23. 2024Panacea: Pareto Alignment via Preference Adaptation for LLMscited by 80
  24. 2024REBEL: Reinforcement Learning via Regressing Relative Rewardscited by 69
  25. 2024Online Iterative Reinforcement Learning from Human Feedback with General Preference Modelcited by 52
  26. 2023Strategic Behavior of Large Language Models: Game Structure vs. Contextual Framingcited by 51
  27. 2024Large Language Models and Cognitive Science: A Comprehensive Review of Similarities, Differences, and Challengescited by 49
  28. 2024Evaluating Language Model Agency through Negotiationscited by 48
  29. 2024Measuring Bargaining Abilities of LLMs: A Benchmark and A Buyer-Enhancement Methodcited by 48
  30. 2024Iterative Nash Policy Optimization: Aligning LLMs with General Preferences via No-Regret Learningcited by 45
  31. 2024Conditional Language Policy: A General Framework for Steerable Multi-Objective Finetuningcited by 43
  32. 2025SPC: Evolving Self-Play Critic via Adversarial Games for LLM Reasoningcited by 42
  33. 2025Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Modelscited by 39
  34. 2025A Survey on Large Language Model Benchmarkscited by 36
  35. 2025The Traitors: Deception and Trust in Multi-Agent Language Model Simulationscited by 34
  36. 2024Provable Pluralistic Alignment: Multi-Party RLHF under Offline Human Feedbackcited by 32
  37. 2025MTSA: Multi-turn Safety Alignment for LLMs through Multi-round Red-teamingcited by 32
  38. 2024Steering Language Models with Game-Theoretic Solverscited by 28

50 most cited of 82 All 82 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Emergent Alignment via Competition
  2. 2026Constructive Alignment: Governing Preference Dynamics in Human-AI Interaction
  3. 2025Multi-level Value Alignment in Agentic AI Systems: Survey and Perspectives
  4. 2025The Coming Crisis of Multi-Agent Misalignment: AI Alignment Must Be a Dynamic and Social Process
  5. 2026Align AI to Dynamic Human-AI Workflows
  6. 2024The AI Alignment Paradox
  7. 2024AI Alignment through Reinforcement Learning from Human Feedback? Contradictions and Limitations
  8. 2026Toward Human-Centered Multi-Agent Systems: Integrating Cognition, Culture, Values, and Cooperation in AI Agents