mlsanityData to 14 Oct 2026

Causal Explanations for Sequential Decision-Making in Multi-Agent Systems

arXiv 2302.10809 ↗First posted 21 Feb 2023cs.AI cs.ROcited by 10

Not on the map. Why?

Abstract (arXiv)

We present CEMA: Causal Explanations in Multi-Agent systems; a framework for creating causal natural language explanations of an agent's decisions in dynamic sequential multi-agent systems to build more trustworthy autonomous agents. Unlike prior work that assumes a fixed causal structure, CEMA only requires a probabilistic model for forward-simulating the state of the system. Using such a model, CEMA simulates counterfactual worlds that identify the salient causes behind the agent's decisions. We evaluate CEMA on the task of motion planning for autonomous driving and test it in diverse simulated scenarios. We show that CEMA correctly and robustly identifies the causes behind the agent's decisions, even when a large number of other agents is present, and show via a user study that CEMA's explanations have a positive effect on participants' trust in autonomous vehicles and are rated as high as high-quality baseline explanations elicited from other participants. We release the collected explanations with annotations as the HEADD dataset.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 22 of its 67 references that are in the map. First posted on arXiv 21 Feb 2023.

Builds on

22 of 67 references.

References · 22

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  2. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
  3. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,089
  4. 2018Metrics for Explainable AI: Challenges and Prospectscited by 260
  5. 2019FACE: Feasible and Actionable Counterfactual Explanationscited by 233
  6. 2020A Survey on the Explainability of Supervised Machine Learningcited by 227
  7. 2018Textual Explanations for Self-Driving Vehiclescited by 226
  8. 2018Programmatically Interpretable Reinforcement Learningcited by 193
  9. 2019Explainable Reinforcement Learning Through a Causal Lenscited by 177
  10. 2017A Survey on Dialogue Systems: Recent Advances and New Frontierscited by 162
  11. 2019Self-Driving Cars: A Surveycited by 148
  12. 2017Autonomous Agents Modelling Other Agents: A Comprehensive Survey and Open Problemscited by 131
10 more
  1. 2021Counterfactual Explanations in Sequential Decision Making Under Uncertaintycited by 37
  2. 2020To Explain or Not to Explain: A Study on the Necessity of Explanations for Autonomous Vehiclescited by 21
  3. 2021Levels of explainable artificial intelligence for human-aligned conversational explanationscited by 21
  4. 2021GRIT: Fast, Interpretable, and Verifiable Goal Recognition with Learned Decision Trees for Autonomous Drivingcited by 19
  5. 2018Foundations of Sequence-to-Sequence Modeling for Time Seriescited by 15
  6. 2021Interpretable Goal Recognition in the Presence of Occluded Factors for Autonomous Vehiclescited by 12
  7. 2022A Human-Centric Method for Generating Causal Explanations in Natural Language for Autonomous Vehicle Motion Planningcited by 7
  8. 2022Verifiable Goal Recognition for Autonomous Driving with Occlusionscited by 7
  9. 2021Understanding Prediction Discrepancies in Machine Learning Classifierscited by 6
  10. 2022Deep Reinforcement Learning for Multi-Agent Interactioncited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 6; 2026: 1.

10 papers.

  1. 2025TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent Systemscited by 59
  2. 2024Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic Reviewcited by 53
  3. 2024Safety Implications of Explainable Artificial Intelligence in End-to-End Autonomous Drivingcited by 7
  4. 2025Unveiling the Black Box: A Multi-Layer Framework for Explaining Reinforcement Learning-Based Cyber Agentscited by 4
  5. 2025Integrating Counterfactual Simulations with Language Models for Explaining Multi-Agent Behaviourcited by 3
  6. 2026Distributed Denial of Science: How Indirect Data Poisoning of AI Systems Can Industrialize Scientific Fraudcited by 2
  7. 2024Extending Structural Causal Models for Autonomous Vehicles to Simplify Temporal System Construction & Enable Dynamic Interactions Between Agentscited by 1
  8. 2025Generating Causal Explanations of Vehicular Agent Behavioural Interactions with Learnt Reward Profilescited by 0
  9. 2025Explaining Autonomous Vehicles with Intention-aware Policy Graphscited by 0
  10. 2025Adaptive Accountability in Networked MAS: Tracing and Mitigating Emergent Norms at Scalecited by 0