mlsanityData to 7 Oct 2026

Incentive-Aware AI Safety via Strategic Resource Allocation: A Stackelberg Security Games Perspective

arXiv 2602.07259 ↗First posted 6 Feb 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

As AI systems grow more capable and autonomous, ensuring their safety and reliability requires not only model-level alignment but also strategic oversight of the humans and institutions involved in their development and deployment. Existing safety frameworks largely treat alignment as a static optimization problem (e.g., tuning models to desired behavior) while overlooking the dynamic, adversarial incentives that shape how data are collected, how models are evaluated, and how they are ultimately deployed. We propose a new perspective on AI safety grounded in Stackelberg Security Games (SSGs): a class of game-theoretic models designed for adversarial resource allocation under uncertainty. By viewing AI oversight as a strategic interaction between defenders (auditors, evaluators, and deployers) and attackers (malicious actors, misaligned contributors, or worst-case failure modes), SSGs provide a unifying framework for reasoning about incentive design, limited oversight capacity, and adversarial uncertainty across the AI lifecycle. We illustrate how this framework can inform (1) training-time auditing against data/feedback poisoning, (2) pre-deployment evaluation under constrained reviewer resources, and (3) robust multi-model deployment in adversarial environments. This synthesis bridges algorithmic alignment and institutional oversight design, highlighting how game-theoretic deterrence can make AI oversight proactive, risk-aware, and resilient to manipulation.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 34 of its 77 references that are in the map. First posted on arXiv 6 Feb 2026.

Builds on

34 of 77 references.

References · 34

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2017Deep reinforcement learning from human preferencescited by 3,846
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  4. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  5. 2024A Survey on Large Language Models for Code Generationcited by 646
  6. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  7. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 337
  8. 2023Are aligned neural networks adversarially aligned?cited by 305
  9. 2023Poisoning Web-Scale Training Datasets is Practicalcited by 276
  10. 2024Frontier Models are Capable of In-context Schemingcited by 252
  11. 2023AI Control: Improving Safety Despite Intentional Subversioncited by 207
  12. 2025Why Language Models Hallucinatecited by 205
22 more
  1. 2024Large Language Models for Education: A Survey and Outlookcited by 185
  2. 2023A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicitycited by 180
  3. 2023Large Language Models in Law: A Surveycited by 128
  4. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  5. 2024Collective Constitutional AI: Aligning a Language Model with Public Inputcited by 120
  6. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  7. 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 81
  8. 2025LLM4SR: A Survey on Large Language Models for Scientific Researchcited by 81
  9. 2024Me, Myself, and AI: The Situational Awareness Dataset (SAD) for LLMscited by 78
  10. 2025Stress Testing Deliberative Alignment for Anti-Scheming Trainingcited by 71
  11. 2022Fine-mixing: Mitigating Backdoors in Fine-tuned Language Modelscited by 50
  12. 2024Persistent Pre-Training Poisoning of LLMscited by 46
  13. 2023Fake Alignment: Are LLMs Really Aligned Well?cited by 39
  14. 2024Large Language Models for Medicine: A Surveycited by 38
  15. 2025A sketch of an AI control safety casecited by 36
  16. 2024Best-of-Venom: Attacking RLHF by Injecting Poisoned Preference Datacited by 33
  17. 2024Noise Injection Reveals Hidden Capabilities of Sandbagging Language Modelscited by 22
  18. 2025LLMs Can Covertly Sandbag on Capability Evaluations Against Chain-of-Thought Monitoringcited by 17
  19. 2024The Dark Side of Human Feedback: Poisoning Large Language Models via User Inputscited by 10
  20. 2025Point Convergence of Nesterov's Accelerated Gradient Method: An AI-Assisted Proofcited by 10
  21. 2025Stackelberg Learning from Human Feedback: Preference Optimization as a Sequential Gamecited by 9
  22. 2025Strategyproof Reinforcement Learning from Human Feedbackcited by 5

Built on by

Not cited yet.