Corrigible Assistance in One Round: Pragmatic-Pedagogic Best Response
Not on the map. Why?
Abstract (arXiv)
Assistance games formalize human-robot collaboration under asymmetric information: the human knows the goal, while the robot must infer it from observation and interaction in order to assist effectively. In general, computing optimal assistance game strategies online is intractable, since exact solutions require planning in a POMDP. We identify a class of assistance games in which pragmatic-pedagogic reasoning resolves goal uncertainty in a single time step, rendering the full-horizon game exactly solvable by a tractable best-response procedure. Within this class, we show that mainstream inverse optimal control exhibits an inference ceiling that hinders alignment, while pragmatic-pedagogic reasoning overcomes this barrier by immediately disambiguating goals through actions that look equivalent under task execution alone. Finally, we validate our theoretical results and proposed method on a simple collaborative block-building example.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 28 references that are in the map. First posted on arXiv 29 Jul 2026.
Builds on
13 of 28 references.
References · 13
- 2017Deep reinforcement learning from human preferencescited by 3,912
- 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 759
- 2016Cooperative Inverse Reinforcement Learningcited by 300
- 2016The Off-Switch Gamecited by 97
- 2024On Targeted Manipulation and Deception when Optimizing LLMs for User Feedbackcited by 47
- 2017Pragmatic-Pedagogic Value Alignmentcited by 36
- 2017Should Robots be Obedient?cited by 20
- 2020LESS is More: Rethinking Probabilistic Models of Human Behaviorcited by 17
- 2024When Your AIs Deceive You: Challenges of Partial Observability in Reinforcement Learning from Human Feedbackcited by 17
- 2019Literal or Pedagogic Human? Analyzing Human Model Misspecification in Objective Learningcited by 15
- 2023Aligning Robot and Human Representationscited by 11
- 2018Goal Inference Improves Objective and Perceived Performance in Human-Robot Collaborationcited by 10
1 more
- 2025AssistanceZero: Scalably Solving Assistance Gamescited by 10
Built on by
Not cited yet.