# Mutual Understanding between People and Systems via Neurosymbolic AI and Knowledge Graphs

arXiv: https://arxiv.org/abs/2504.11200 · HTML: https://mlsanity.com/paper/2504.11200/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 86 references that are in the map.
- First posted on arXiv 15 Apr 2025.

## Builds on (most cited first, 25 of 25)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [CogVLM: Visual Expert for Pretrained Language Models](https://mlsanity.com/paper/2311.03079/) (2023)
- [SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities](https://mlsanity.com/paper/2401.12168/) (2024)
- [Compositional Chain-of-Thought Prompting for Large Multimodal Models](https://mlsanity.com/paper/2311.17076/) (2023)
- [Few-NERD: A Few-Shot Named Entity Recognition Dataset](https://mlsanity.com/paper/2105.07464/) (2021)
- [Can Large Language Models Reason and Plan?](https://mlsanity.com/paper/2403.04121/) (2024)
- [Croissant: A Metadata Format for ML-Ready Datasets](https://mlsanity.com/paper/2403.19546/) (2024)
- [Thinking Fast and Slow in AI](https://mlsanity.com/paper/2010.06002/) (2020)
- [Modular Design Patterns for Hybrid Learning and Reasoning Systems: a taxonomy, patterns and use cases](https://mlsanity.com/paper/2102.11965/) (2021)
- [Packaging research artefacts with RO-Crate](https://mlsanity.com/paper/2108.06503/) (2021)
- [A Boxology of Design Patterns for Hybrid Learning and Reasoning Systems](https://mlsanity.com/paper/1905.12389/) (2019)
- [Generative Multimodal Entity Linking](https://mlsanity.com/paper/2306.12725/) (2023)
- [Architecture Disentanglement for Deep Neural Networks](https://mlsanity.com/paper/2003.13268/) (2020)
- [Submitting surveys via a conversational interface: an evaluation of user acceptance and approach effectiveness](https://mlsanity.com/paper/2003.02537/) (2020)
- [Ontological Smart Contracts in OASIS: Ontology for Agents, Systems, and Integration of Services (Extended Version)](https://mlsanity.com/paper/2012.01410/) (2020)
- [Human Evaluation of Procedural Knowledge Graph Extraction from Text with Large Language Models](https://mlsanity.com/paper/2412.03589/) (2024)
- [Turning Transport Data to Comply with EU Standards while Enabling a Multimodal Transport Knowledge Graph](https://mlsanity.com/paper/2011.06423/) (2020)
- [AI for Low-Code for AI](https://mlsanity.com/paper/2305.20015/) (2023)
- [Towards a Framework for Visual Intelligence in Service Robotics: Epistemic Requirements and Gap Analysis](https://mlsanity.com/paper/2003.06171/) (2020)
- [Procedural Text Mining with Large Language Models](https://mlsanity.com/paper/2310.03376/) (2023)
- [A Framework to build Games with a Purpose for Linked Data Refinement](https://mlsanity.com/paper/1811.02848/) (2018)
- [Interplay of Game Incentives, Player Profiles and Task Difficulty in Games with a Purpose](https://mlsanity.com/paper/1811.08244/) (2018)
- [Modelling Business Agreements in the Multimodal Transportation Domain through Ontological Smart Contracts](https://mlsanity.com/paper/2209.05463/) (2022)
- [Intelligent Urban Traffic Management via Semantic Interoperability across Multiple Heterogeneous Mobility Data Sources](https://mlsanity.com/paper/2407.10539/) (2024)

## Built on by (most cited first, 1 of 1)

- [Knowledge Affordances for Hybrid Human-AI Information Seeking](https://mlsanity.com/paper/2604.27539/) (2026)

## Abstract (arXiv)

This chapter investigates the concept of mutual understanding between humans and systems, positing that Neuro-symbolic Artificial Intelligence (NeSy AI) methods can significantly enhance this mutual understanding by leveraging explicit symbolic knowledge representations with data-driven learning models. We start by introducing three critical dimensions to characterize mutual understanding: sharing knowledge, exchanging knowledge, and governing knowledge. Sharing knowledge involves aligning the conceptual models of different agents to enable a shared understanding of the domain of interest. Exchanging knowledge relates to ensuring the effective and accurate communication between agents. Governing knowledge concerns establishing rules and processes to regulate the interaction between agents. Then, we present several different use case scenarios that demonstrate the application of NeSy AI and Knowledge Graphs to aid meaningful exchanges between human, artificial, and robotic agents. These scenarios highlight both the potential and the challenges of combining top-down symbolic reasoning with bottom-up neural learning, guiding the discussion of the coverage provided by current solutions along the dimensions of sharing, exchanging, and governing knowledge. Concurrently, this analysis facilitates the identification of gaps and less developed aspects in mutual understanding to address in future research.