mlsanityData to 14 Oct 2026

Mechanism Design for Alignment and Control

arXiv 2609.01595 ↗First posted 1 Sep 2026econ.TH cs.AI cs.GTcited by 2

Not on the map. Why?

Abstract (arXiv)

We develop a framework for mechanism design with AI agents whose alignment (preferences) and capabilities (feasible actions and information) are unknown. We want such agents to act on our behalf so mechanisms must incentivize both honesty and obedience. A one-sided imitation structure---capabilities can be concealed but not counterfeited---yields a revelation principle, a characterization of implementable policies via nested cyclical monotonicity, and conditions under which eliciting higher-order beliefs can discipline multiple agents. We apply our framework to stylized examples of (i) sandbagging in which a more capable agent pretends to be less capable; (ii) an alignment--interpretability trade-off, where the two are substitutes in the instrument but complements in value; (iii) discipline via peer scoring; (iv) coupling rewards to induce competition among multiple agents; and (v) scalable oversight and reward shaping.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 35 of its 88 references that are in the map. First posted on arXiv 1 Sep 2026.

Builds on

35 of 88 references.

References · 35

  1. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  2. 2024OpenAI o1 System Cardcited by 1,244
  3. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 529
  4. 2018Scalable agent alignment via reward modeling: a research directioncited by 447
  5. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 384
  6. 2024Alignment faking in large language modelscited by 360
  7. 2024Frontier Models are Capable of In-context Schemingcited by 287
  8. 2019Risks from Learned Optimization in Advanced Machine Learning Systemscited by 258
  9. 2025Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetycited by 249
  10. 2022Measuring Progress on Scalable Oversight for Large Language Modelscited by 225
  11. 2025Open Problems in Mechanistic Interpretabilitycited by 179
  12. 2024Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Modelscited by 178
23 more
  1. 2023Model evaluation for extreme riskscited by 177
  2. 2025Multi-Agent Risks from Advanced AIcited by 163
  3. 2025Measuring AI Ability to Complete Long Software Taskscited by 156
  4. 2018Supervising strong learners by amplifying weak expertscited by 145
  5. 2025Progent: Securing AI Agents with Privilege Controlcited by 141
  6. 2024Evaluating Frontier Models for Dangerous Capabilitiescited by 128
  7. 2025Large Language Models Often Know When They Are Being Evaluatedcited by 128
  8. 2022Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goalscited by 126
  9. 2025Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIscited by 67
  10. 2024Adaptive Deployment of Untrusted LLMs Reduces Distributed Threatscited by 31
  11. 2024Noise Injection Reveals Hidden Capabilities of Sandbagging Language Modelscited by 24
  12. 2025The Elicitation Game: Evaluating Capability Elicitation Techniquescited by 18
  13. 2024Risk thresholds for frontier AIcited by 9
  14. 2026Predicting LLM Safety Before Release by Simulating Deploymentcited by 8
  15. 2025Emergent Alignment via Competitioncited by 5
  16. 2026Debate Training Reduces Reward Hacking in RLAIFcited by 4
  17. 2025Friend or Foe: Delegating to an AI Whose Alignment is Unknowncited by 2
  18. 2026Exploration Hacking: Can LLMs Learn to Resist RL Training?cited by 2
  19. 2026Screening for Choice Setscited by 1
  20. 2026Robust Trustcited by 1
  21. 2026Realistic honeypot evaluations for scheming propensitycited by 1
  22. 2026Bootstrapped Monitoring: Leveraging Transparent Reasoning to Oversee Stronger AI Agentscited by 1
  23. 2026Radical AI Interpretabilitycited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.