Experiments with Detecting and Mitigating AI Deception
Not on the map. Why?
Abstract (arXiv)
How to detect and mitigate deceptive AI systems is an open problem for the field of safe and trustworthy AI. We analyse two algorithms for mitigating deception: The first is based on the path-specific objectives framework where paths in the game that incentivise deception are removed. The second is based on shielding, i.e., monitoring for unsafe policies and replacing them with a safe reference policy. We construct two simple games and evaluate our algorithms empirically. We find that both methods ensure that our agent is not deceptive, however, shielding tends to achieve higher reward.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 17 references that are in the map. First posted on arXiv 26 Jun 2023.
Builds on
13 of 17 references.
References · 13
- 2023GPT-4 Technical Reportcited by 18,396
- 2017Safe Reinforcement Learning via Shieldingcited by 449
- 2022Is Power-Seeking AI an Existential Risk?cited by 105
- 2021Safe Multi-Agent Reinforcement Learning via Shieldingcited by 61
- 2022Safe Reinforcement Learning via Shielding under Partial Observabilitycited by 56
- 2018Towards Formal Definitions of Blameworthiness, Intention, and Moral Responsibilitycited by 32
- 2022Discovering Agentscited by 28
- 2022Path-Specific Objectives for Safer Agent Incentivescited by 20
- 2023Reasoning about Causality in Gamescited by 20
- 2021Shielding Atari Games with Bounded Presciencecited by 14
- 2022Correct-by-Construction Runtime Enforcement in AI -- A Surveycited by 12
- 2022Dynamic Shielding for Reinforcement Learning in Black-Box Environmentscited by 11
1 more
- 2021Definitions of intent suitable for algorithmscited by 8
Built on by
Not cited yet.