# Symbols as a Lingua Franca for Bridging Human-AI Chasm for Explainable and Advisable AI Systems

arXiv: https://arxiv.org/abs/2109.09904 · HTML: https://mlsanity.com/paper/2109.09904/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 32 references that are in the map.
- First posted on arXiv 21 Sep 2021.

## Builds on (most cited first, 11 of 11)

- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Neural-Symbolic Computing: An Effective Methodology for Principled Integration of Machine Learning and Reasoning](https://mlsanity.com/paper/1905.06088/) (2019)
- [Visualizing and Understanding Atari Agents](https://mlsanity.com/paper/1711.00138/) (2017)
- [The Off-Switch Game](https://mlsanity.com/paper/1611.08219/) (2016)
- [Foundations for Restraining Bolts: Reinforcement Learning with LTLf/LDLf restraining specifications](https://mlsanity.com/paper/1807.06333/) (2018)
- [Widening the Pipeline in Human-Guided Reinforcement Learning with Explanation and Context-Aware Data Augmentation](https://mlsanity.com/paper/2006.14804/) (2020)
- [Learning First-Order Symbolic Representations for Planning from the Structure of the State Space](https://mlsanity.com/paper/1909.05546/) (2019)
- [Bridging the Gap: Providing Post-Hoc Symbolic Explanations for Sequential Decision-Making Problems with Inscrutable Representations](https://mlsanity.com/paper/2002.01080/) (2020)
- [Challenges of Human-Aware AI Systems](https://mlsanity.com/paper/1910.07089/) (2019)

## Built on by (most cited first, 30 of 36)

- [Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning](https://mlsanity.com/paper/2305.14909/) (2023)
- [GlanceNets: Interpretabile, Leak-proof Concept-based Models](https://mlsanity.com/paper/2205.15612/) (2022)
- [Leveraging Explanations in Interactive Machine Learning: An Overview](https://mlsanity.com/paper/2207.14526/) (2022)
- [BEARS Make Neuro-Symbolic Models Aware of their Reasoning Shortcuts](https://mlsanity.com/paper/2402.12240/) (2024)
- [Neuro-Symbolic Continual Learning: Knowledge, Reasoning Shortcuts and Concept Rehearsal](https://mlsanity.com/paper/2302.01242/) (2023)
- [Interpretable Concept Bottlenecks to Align Reinforcement Learning Agents](https://mlsanity.com/paper/2401.05821/) (2024)
- [Leveraging Approximate Symbolic Models for Reinforcement Learning via Skill Diversity](https://mlsanity.com/paper/2202.02886/) (2022)
- [Interpretability is in the Mind of the Beholder: A Causal Framework for Human-interpretable Representation Learning](https://mlsanity.com/paper/2309.07742/) (2023)
- [A Neuro-Symbolic Benchmark Suite for Concept Quality and Reasoning Shortcuts](https://mlsanity.com/paper/2406.10368/) (2024)
- [Boosting Object Representation Learning via Motion and Object Continuity](https://mlsanity.com/paper/2211.09771/) (2022)
- [Neural Concept Binder](https://mlsanity.com/paper/2406.09949/) (2024)
- [Challenges in Human-Agent Communication](https://mlsanity.com/paper/2412.10380/) (2024)
- [HackAtari: Atari Learning Environments for Robust and Continual Reinforcement Learning](https://mlsanity.com/paper/2406.03997/) (2024)
- [Symbol Guided Hindsight Priors for Reward Learning from Human Preferences](https://mlsanity.com/paper/2210.09151/) (2022)
- [Relative Behavioral Attributes: Filling the Gap between Symbolic Goal Specification and Reward Learning from Human Preferences](https://mlsanity.com/paper/2210.15906/) (2022)
- [Egocentric Planning for Scalable Embodied Task Achievement](https://mlsanity.com/paper/2306.01295/) (2023)
- [Formal Explanations for Neuro-Symbolic AI](https://mlsanity.com/paper/2410.14219/) (2024)
- [Learning from Ambiguous Demonstrations with Self-Explanation Guided Reinforcement Learning](https://mlsanity.com/paper/2110.05286/) (2021)
- [Towards Effective and Interpretable Human-Agent Collaboration in MOBA Games: A Communication Perspective](https://mlsanity.com/paper/2304.11632/) (2023)
- [How Faithful are Self-Explainable GNNs?](https://mlsanity.com/paper/2308.15096/) (2023)
- [Towards customizable reinforcement learning agents: Enabling preference specification through online vocabulary expansion](https://mlsanity.com/paper/2210.15096/) (2022)
- [Exploiting Unlabeled Data for Feedback Efficient Human Preference based Reinforcement Learning](https://mlsanity.com/paper/2302.08738/) (2023)
- [PlaSma: Making Small Language Models Better Procedural Knowledge Models for (Counterfactual) Planning](https://mlsanity.com/paper/2305.19472/) (2023)
- [Online inductive learning from answer sets for efficient reinforcement learning exploration](https://mlsanity.com/paper/2501.07445/) (2025)
- [Integrating AI Planning with Natural Language Processing: A Combination of Explicit and Tacit Knowledge](https://mlsanity.com/paper/2202.07138/) (2022)
- [Data Driven Reward Initialization for Preference based Reinforcement Learning](https://mlsanity.com/paper/2302.08733/) (2023)
- [Fodor and Pylyshyn's Legacy: Still No Human-like Systematic Compositionality in Neural Networks](https://mlsanity.com/paper/2506.01820/) (2025)
- [A State Augmentation based approach to Reinforcement Learning from Human Preferences](https://mlsanity.com/paper/2302.08734/) (2023)
- [Rethinking the Evaluating Framework for Natural Language Understanding in AI Systems: Language Acquisition as a Core for Future Metrics](https://mlsanity.com/paper/2309.11981/) (2023)
- [Goal Alignment: A Human-Aware Account of Value Alignment Problem](https://mlsanity.com/paper/2302.00813/) (2023)

## Abstract (arXiv)

Despite the surprising power of many modern AI systems that often learn their own representations, there is significant discontent about their inscrutability and the attendant problems in their ability to interact with humans. While alternatives such as neuro-symbolic approaches have been proposed, there is a lack of consensus on what they are about. There are often two independent motivations (i) symbols as a lingua franca for human-AI interaction and (ii) symbols as system-produced abstractions used by the AI system in its internal reasoning. The jury is still out on whether AI systems will need to use symbols in their internal reasoning to achieve general intelligence capabilities. Whatever the answer there is, the need for (human-understandable) symbols in human-AI interaction seems quite compelling. Symbols, like emotions, may well not be sine qua non for intelligence per se, but they will be crucial for AI systems to interact with us humans -- as we can neither turn off our emotions nor get by without our symbols. In particular, in many human-designed domains, humans would be interested in providing explicit (symbolic) knowledge and advice -- and expect machine explanations in kind. This alone requires AI systems to to maintain a symbolic interface for interaction with humans. In this blue sky paper, we argue this point of view, and discuss research directions that need to be pursued to allow for this type of human-AI interaction.