Why did I fail? A Causal-based Method to Find Explanations for Robot Failures
Not on the map. Why?
Abstract (arXiv)
Robot failures in human-centered environments are inevitable. Therefore, the ability of robots to explain such failures is paramount for interacting with humans to increase trust and transparency. To achieve this skill, the main challenges addressed in this paper are I) acquiring enough data to learn a cause-effect model of the environment and II) generating causal explanations based on that model. We address I) by learning a causal Bayesian network from simulation data. Concerning II), we propose a novel method that enables robots to generate contrastive explanations upon task failures. The explanation is based on setting the failure state in contrast with the closest state that would have allowed for a successful execution. This state is found through breadth-first search and is based on success predictions from the learned causal model. We assessed our method in two different scenarios I) stacking cubes and II) dropping spheres into a container. The obtained causal models reach a sim2real accuracy of 70% and 72%, respectively. We finally show that our novel method scales over multiple tasks and allows real robots to give failure explanations like 'the upper cube was stacked too high and too far to the right of the lower cube.'
Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 13 of its 31 references that are in the map. First posted on arXiv 9 Apr 2022.
Builds on
13 of 31 references.
References · 13
- 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
- 2018DAGs with NO TEARS: Continuous Optimization for Structure Learningcited by 745
- 2017Neural Motifs: Scene Graph Parsing with Global Contextcited by 413
- 2021D'ya like DAGs? A Survey on Structure Learning and Causal Discoverycited by 209
- 2020The Emerging Landscape of Explainable AI Planning and Decision Makingcited by 44
- 2021Explainable AI for Robot Failures: Generating Explanations that Improve User Assistance in Fault Recoverycited by 32
- 2021Causal Reasoning in Simulation for Structure and Transfer Learning of Robot Manipulation Policiescited by 20
- 2021DoWhy: Addressing Challenges in Expressing and Validating Causal Assumptionscited by 20
- 2020Review of The Book of Why: The New Science of Cause and Effectcited by 19
- 2021Semantic-Based Explainable AI: Leveraging Semantic Scene Graphs and Pairwise Ranking to Explain Robot Failurescited by 12
- 2015Learning Discrete Bayesian Networks from Continuous Datacited by 6
- 2021Work in Progress -- Automated Generation of Robotic Planning Domains from Observationscited by 2
1 more
Built on by
Citing papers per year: 2023: 2; 2024: 4; 2025: 7.
13 papers.
Sort:
By area:
- 2023REFLECT: Summarizing Robot Experiences for Failure Explanation and Correctioncited by 171
- 2024Automating Robot Failure Recovery Using Vision-Language Models With Optimized Promptscited by 16
- 2023CAR-DESPOT: Causally-Informed Online POMDP Planning for Robots in Confounded Environmentscited by 11
- 2024CURE: Simulation-Augmented Auto-Tuning in Roboticscited by 2
- 2025STAR: A Foundation Model-driven Framework for Robust Task Planning and Failure Recovery in Robotic Systemscited by 2
- 2025Exploring Causality for HRI: A Case Study on Robotic Mental Well-being Coachingcited by 2
- 2025Hierarchical Vision Language Action Model Using Success and Failure Demonstrationscited by 2
- 2024Multimodal Coherent Explanation Generation of Robot Failurescited by 1
- 2025Learning Causal Structure Distributions for Robust Planningcited by 1
- 2024COBRA-PPM: A Causal Bayesian Reasoning Architecture Using Probabilistic Programming for Robot Manipulation Under Uncertaintycited by 0
- 2025Robot Pouring: Identifying Causes of Spillage and Selecting Alternative Action Parameters Using Probabilistic Actual Causationcited by 0
- 2025Generating Causal Explanations of Vehicular Agent Behavioural Interactions with Learnt Reward Profilescited by 0
- 2025Hierarchical Vision Language Action Model Using Success and Failure Demonstrations3 Dec 2025
- 2025Learning Causal Structure Distributions for Robust Planning8 Aug 2025
- 2025Adapting Robot's Explanation for Failures Based on Observed Human Behavior in Human-Robot Collaboration13 Apr 2025
- 2025Generating Causal Explanations of Vehicular Agent Behavioural Interactions with Learnt Reward Profiles18 Mar 2025
- 2025STAR: A Foundation Model-driven Framework for Robust Task Planning and Failure Recovery in Robotic Systems8 Mar 2025
- 2025Exploring Causality for HRI: A Case Study on Robotic Mental Well-being Coaching4 Mar 2025
- 2025Robot Pouring: Identifying Causes of Spillage and Selecting Alternative Action Parameters Using Probabilistic Actual Causation13 Feb 2025
- 2024Multimodal Coherent Explanation Generation of Robot Failures1 Oct 2024
- 2024Automating Robot Failure Recovery Using Vision-Language Models With Optimized Prompts6 Sep 2024
- 2024COBRA-PPM: A Causal Bayesian Reasoning Architecture Using Probabilistic Programming for Robot Manipulation Under Uncertainty21 Mar 2024
- 2024CURE: Simulation-Augmented Auto-Tuning in Robotics8 Feb 2024
- 2023REFLECT: Summarizing Robot Experiences for Failure Explanation and Correction27 Jun 2023
- 2023REFLECT: Summarizing Robot Experiences for Failure Explanation and Correctioncited by 171
- 2023CAR-DESPOT: Causally-Informed Online POMDP Planning for Robots in Confounded Environmentscited by 11
- 2024CURE: Simulation-Augmented Auto-Tuning in Roboticscited by 2
- 2025STAR: A Foundation Model-driven Framework for Robust Task Planning and Failure Recovery in Robotic Systemscited by 2
- 2025Exploring Causality for HRI: A Case Study on Robotic Mental Well-being Coachingcited by 2
- 2024Multimodal Coherent Explanation Generation of Robot Failurescited by 1
- 2025Learning Causal Structure Distributions for Robust Planningcited by 1
- 2024COBRA-PPM: A Causal Bayesian Reasoning Architecture Using Probabilistic Programming for Robot Manipulation Under Uncertaintycited by 0
- 2025Robot Pouring: Identifying Causes of Spillage and Selecting Alternative Action Parameters Using Probabilistic Actual Causationcited by 0
- 2025Adapting Robot's Explanation for Failures Based on Observed Human Behavior in Human-Robot Collaborationcited by 0
- 2025Learning Causal Structure Distributions for Robust Planning8 Aug 2025
- 2025Adapting Robot's Explanation for Failures Based on Observed Human Behavior in Human-Robot Collaboration13 Apr 2025
- 2025STAR: A Foundation Model-driven Framework for Robust Task Planning and Failure Recovery in Robotic Systems8 Mar 2025
- 2025Exploring Causality for HRI: A Case Study on Robotic Mental Well-being Coaching4 Mar 2025
- 2025Robot Pouring: Identifying Causes of Spillage and Selecting Alternative Action Parameters Using Probabilistic Actual Causation13 Feb 2025
- 2024Multimodal Coherent Explanation Generation of Robot Failures1 Oct 2024
- 2024COBRA-PPM: A Causal Bayesian Reasoning Architecture Using Probabilistic Programming for Robot Manipulation Under Uncertainty21 Mar 2024
- 2024CURE: Simulation-Augmented Auto-Tuning in Robotics8 Feb 2024
- 2023REFLECT: Summarizing Robot Experiences for Failure Explanation and Correction27 Jun 2023
- 2023CAR-DESPOT: Causally-Informed Online POMDP Planning for Robots in Confounded Environments13 Apr 2023