# Analyzing Intentional Behavior in Autonomous Agents under Uncertainty

arXiv: https://arxiv.org/abs/2307.01532 · HTML: https://mlsanity.com/paper/2307.01532/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 66 references that are in the map.
- First posted on arXiv 4 Jul 2023.

## Builds on (most cited first, 13 of 13)

- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [Explainable Reinforcement Learning: A Survey](https://mlsanity.com/paper/2005.06247/) (2020)
- [The Probabilistic Model Checker Storm](https://mlsanity.com/paper/2002.07080/) (2020)
- [Towards Formal Definitions of Blameworthiness, Intention, and Moral Responsibility](https://mlsanity.com/paper/1810.05903/) (2018)
- [Counterfactual harm](https://mlsanity.com/paper/2204.12993/) (2022)
- [A Causal Analysis of Harm](https://mlsanity.com/paper/2210.05327/) (2022)
- [TEMPEST -- Synthesis Tool for Reactive Systems and Shields in Probabilistic Environments](https://mlsanity.com/paper/2105.12588/) (2021)
- [On Blame Attribution for Accountable Multi-Agent Sequential Decision Making](https://mlsanity.com/paper/2107.11927/) (2021)
- [The Computational Complexity of Structure-Based Causality](https://mlsanity.com/paper/1412.3076/) (2014)
- [dtControl: Decision Tree Learning Algorithms for Controller Representation](https://mlsanity.com/paper/2002.04991/) (2020)
- ['Put the Car on the Stand': SMT-based Oracles for Investigating Decisions](https://mlsanity.com/paper/2305.05731/) (2023)

## Built on by (most cited first, 2 of 2)

- [Beyond the Individual: Introducing Group Intention Forecasting with SHOT Dataset](https://mlsanity.com/paper/2509.20715/) (2025)
- [dtControl2+varepsilon: Trading Optimality for Explainability in MDPs via Decision Trees](https://mlsanity.com/paper/2607.25925/) (2026)

## Abstract (arXiv)

Principled accountability for autonomous decision-making in uncertain environments requires distinguishing intentional outcomes from negligent designs from actual accidents. We propose analyzing the behavior of autonomous agents through a quantitative measure of the evidence of intentional behavior. We model an uncertain environment as a Markov Decision Process (MDP). For a given scenario, we rely on probabilistic model checking to compute the ability of the agent to influence reaching a certain event. We call this the scope of agency. We say that there is evidence of intentional behavior if the scope of agency is high and the decisions of the agent are close to being optimal for reaching the event. Our method applies counterfactual reasoning to automatically generate relevant scenarios that can be analyzed to increase the confidence of our assessment. In a case study, we show how our method can distinguish between 'intentional' and 'accidental' traffic collisions.