# Explainable Autonomous Robots: A Survey and Perspective

arXiv: https://arxiv.org/abs/2105.02658 · HTML: https://mlsanity.com/paper/2105.02658/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 94 references that are in the map.
- First posted on arXiv 6 May 2021.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Recurrent World Models Facilitate Policy Evolution](https://mlsanity.com/paper/1809.01999/) (2018)
- [Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models](https://mlsanity.com/paper/1708.08296/) (2017)
- [Programmatically Interpretable Reinforcement Learning](https://mlsanity.com/paper/1804.02477/) (2018)
- [Explainable Reinforcement Learning Through a Causal Lens](https://mlsanity.com/paper/1905.10958/) (2019)
- [Explainable Reinforcement Learning: A Survey](https://mlsanity.com/paper/2005.06247/) (2020)
- [State Representation Learning for Control: An Overview](https://mlsanity.com/paper/1802.04181/) (2018)
- [Automated Rationale Generation: A Technique for Explainable AI and its Effects on Human Perceptions](https://mlsanity.com/paper/1901.03729/) (2019)
- [World Model as a Graph: Learning Latent Landmarks for Planning](https://mlsanity.com/paper/2011.12491/) (2020)
- [Interestingness Elements for Explainable Reinforcement Learning: Understanding Agents' Capabilities and Limitations](https://mlsanity.com/paper/1912.09007/) (2019)
- [Grader variability and the importance of reference standards for evaluating machine learning models for diabetic retinopathy](https://mlsanity.com/paper/1710.01711/) (2017)
- [Contrastive Explanations for Reinforcement Learning in terms of Expected Consequences](https://mlsanity.com/paper/1807.08706/) (2018)
- [The Emerging Landscape of Explainable AI Planning and Decision Making](https://mlsanity.com/paper/2002.11697/) (2020)
- [Learning Finite State Representations of Recurrent Policy Networks](https://mlsanity.com/paper/1811.12530/) (2018)
- [Enabling Robots to Communicate their Objectives](https://mlsanity.com/paper/1702.03465/) (2017)
- [Explainability in Human-Agent Systems](https://mlsanity.com/paper/1904.08123/) (2019)
- [Explainable AI for Robot Failures: Generating Explanations that Improve User Assistance in Fault Recovery](https://mlsanity.com/paper/2101.01625/) (2021)
- [Symbol Emergence in Robotics: A Survey](https://mlsanity.com/paper/1509.08973/) (2015)
- [Explicability? Legibility? Predictability? Transparency? Privacy? Security? The Emerging Landscape of Interpretable Agent Behavior](https://mlsanity.com/paper/1811.09722/) (2018)
- [Joint Mind Modeling for Explanation Generation in Complex Human-Robot Collaborative Tasks](https://mlsanity.com/paper/2007.12803/) (2020)
- [Distal Explanations for Model-free Explainable Reinforcement Learning](https://mlsanity.com/paper/2001.10284/) (2020)
- [Autonomous Self-Explanation of Behavior for Interactive Reinforcement Learning Agents](https://mlsanity.com/paper/1810.08811/) (2018)
- [Designing Environments Conducive to Interpretable Robot Behavior](https://mlsanity.com/paper/2007.00820/) (2020)
- [Sequential Explanations with Mental Model-Based Policies](https://mlsanity.com/paper/2007.09028/) (2020)

## Built on by (most cited first, 15 of 15)

- [Trust in AI: Progress, Challenges, and Future Directions](https://mlsanity.com/paper/2403.14680/) (2024)
- [DreamingV2: Reinforcement Learning with Discrete World Models without Reconstruction](https://mlsanity.com/paper/2203.00494/) (2022)
- [Episodic memory in AI agents poses risks that should be studied and mitigated](https://mlsanity.com/paper/2501.11739/) (2025)
- [Distilling Reinforcement Learning Policies for Interpretable Robot Locomotion: Gradient Boosting Machines and Symbolic Regression](https://mlsanity.com/paper/2403.14328/) (2024)
- [Explain yourself! Effects of Explanations in Human-Robot Interaction](https://mlsanity.com/paper/2204.04501/) (2022)
- [Enhancing Trust in Autonomous Agents: An Architecture for Accountability and Explainability through Blockchain and Large Language Models](https://mlsanity.com/paper/2403.09567/) (2024)
- [Modifying RL Policies with Imagined Actions: How Predictable Policies Can Enable Users to Perform Novel Tasks](https://mlsanity.com/paper/2312.05991/) (2023)
- [Vehicle Behavior Prediction by Episodic-Memory Implanted NDT](https://mlsanity.com/paper/2402.08423/) (2024)
- [Imagining In-distribution States: How Predictable Robot Behavior Can Enable User Control Over Learned Policies](https://mlsanity.com/paper/2406.13711/) (2024)
- [Enhancing Robot Assistive Behaviour with Reinforcement Learning and Theory of Mind](https://mlsanity.com/paper/2411.07003/) (2024)
- [Joint Action Language Modelling for Transparent Policy Execution](https://mlsanity.com/paper/2504.10055/) (2025)
- [Effective Explanations for Belief-Desire-Intention Robots: When and What to Explain](https://mlsanity.com/paper/2507.02016/) (2025)
- [From Facts to Foils: Designing and Evaluating Counterfactual Explanations for Smart Environments](https://mlsanity.com/paper/2510.03078/) (2025)
- [Explaining Why Things Go Where They Go: Interpretable Constructs of Human Organizational Preferences](https://mlsanity.com/paper/2512.24829/) (2025)
- [PAANI : On Device Visual Evidence Fusion and Explainable Guidance for River Robot Simulation](https://mlsanity.com/paper/2609.22353/) (2026)

## Abstract (arXiv)

Advanced communication protocols are critical to enable the coexistence of autonomous robots with humans. Thus, the development of explanatory capabilities is an urgent first step toward autonomous robots. This survey provides an overview of the various types of "explainability" discussed in machine learning research. Then, we discuss the definition of "explainability" in the context of autonomous robots (i.e., explainable autonomous robots) by exploring the question "what is an explanation?" We further conduct a research survey based on this definition and present some relevant topics for future research.