# Corrigible Assistance in One Round: Pragmatic-Pedagogic Best Response

arXiv: https://arxiv.org/abs/2607.27508 · HTML: https://mlsanity.com/paper/2607.27508/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 28 references that are in the map.
- First posted on arXiv 29 Jul 2026.

## Builds on (most cited first, 13 of 13)

- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2307.15217/) (2023)
- [Cooperative Inverse Reinforcement Learning](https://mlsanity.com/paper/1606.03137/) (2016)
- [The Off-Switch Game](https://mlsanity.com/paper/1611.08219/) (2016)
- [On Targeted Manipulation and Deception when Optimizing LLMs for User Feedback](https://mlsanity.com/paper/2411.02306/) (2024)
- [Pragmatic-Pedagogic Value Alignment](https://mlsanity.com/paper/1707.06354/) (2017)
- [Should Robots be Obedient?](https://mlsanity.com/paper/1705.09990/) (2017)
- [LESS is More: Rethinking Probabilistic Models of Human Behavior](https://mlsanity.com/paper/2001.04465/) (2020)
- [When Your AIs Deceive You: Challenges of Partial Observability in Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2402.17747/) (2024)
- [Literal or Pedagogic Human? Analyzing Human Model Misspecification in Objective Learning](https://mlsanity.com/paper/1903.03877/) (2019)
- [Aligning Robot and Human Representations](https://mlsanity.com/paper/2302.01928/) (2023)
- [Goal Inference Improves Objective and Perceived Performance in Human-Robot Collaboration](https://mlsanity.com/paper/1802.01780/) (2018)
- [AssistanceZero: Scalably Solving Assistance Games](https://mlsanity.com/paper/2504.07091/) (2025)

## Abstract (arXiv)

Assistance games formalize human-robot collaboration under asymmetric information: the human knows the goal, while the robot must infer it from observation and interaction in order to assist effectively. In general, computing optimal assistance game strategies online is intractable, since exact solutions require planning in a POMDP. We identify a class of assistance games in which pragmatic-pedagogic reasoning resolves goal uncertainty in a single time step, rendering the full-horizon game exactly solvable by a tractable best-response procedure. Within this class, we show that mainstream inverse optimal control exhibits an inference ceiling that hinders alignment, while pragmatic-pedagogic reasoning overcomes this barrier by immediately disambiguating goals through actions that look equivalent under task execution alone. Finally, we validate our theoretical results and proposed method on a simple collaborative block-building example.