Higher-Order Belief in Incomplete Information MAIDs
Not on the map. Why?
Abstract (arXiv)
Multi-agent influence diagrams (MAIDs) are probabilistic graphical models which represent strategic interactions between agents. MAIDs are equivalent to extensive form games (EFGs) but have a more compact and informative structure. However, MAIDs cannot, in general, represent settings of incomplete information -- wherein agents have different beliefs about the game being played, and different beliefs about each-other's beliefs. In this paper, we introduce incomplete information MAIDs (II-MAIDs). We define both infinite and finite-depth II-MAIDs and prove an equivalence relation to EFGs with incomplete information and no common prior over types. We prove that II-MAIDs inherit classical equilibria concepts via this equivalence, but note that these solution concepts are often unrealistic in the setting with no common prior because they violate common knowledge of rationality. We define a more realistic solution concept based on recursive best-response. Throughout, we describe an example with a hypothetical AI agent undergoing evaluation to illustrate the applicability of II-MAIDs.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 9 of its 38 references that are in the map. First posted on arXiv 8 Mar 2025.
Builds on
9 of 38 references.
References · 9
- 2019Reward Tampering Problems and Solutions in Reinforcement Learning: A Causal Influence Diagram Perspectivecited by 108
- 2021Agent Incentives: A Causal Perspectivecited by 51
- 2023Honesty Is the Best Policy: Defining and Mitigating AI Deceptioncited by 40
- 2024The Reasons that Agents Act: Intention and Instrumental Goalscited by 23
- 2022Path-Specific Objectives for Safer Agent Incentivescited by 20
- 2025MONA: Myopic Optimization with Non-myopic Approval Can Mitigate Multi-step Reward Hackingcited by 19
- 2021Equilibrium Refinements for Multi-Agent Influence Diagrams: Theory and Practicecited by 9
- 2014Networks of Influence Diagrams: A Formalism for Representing Agents' Beliefs and Decision-Making Processescited by 4
- 2020Incentives for Responsiveness, Instrumental Control and Impactcited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026The Impossibility of Eliciting Latent Knowledgecited by 1
- 2026The Impossibility of Eliciting Latent Knowledge10 Jun 2026