# Reasoning with the Theory of Mind for Pragmatic Semantic Communication

arXiv: https://arxiv.org/abs/2311.18224 · HTML: https://mlsanity.com/paper/2311.18224/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 23 references that are in the map.
- First posted on arXiv 30 Nov 2023.

## Builds on (most cited first, 15 of 15)

- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Deep Learning Enabled Semantic Communication Systems](https://mlsanity.com/paper/2006.10685/) (2020)
- [6G Networks: Beyond Shannon Towards Semantic and Goal-Oriented Communications](https://mlsanity.com/paper/2011.14844/) (2020)
- [Less Data, More Knowledge: Building Next Generation Semantic Communication Networks](https://mlsanity.com/paper/2211.14343/) (2022)
- [DeepJSCC-Q: Constellation Constrained Deep Joint Source-Channel Coding](https://mlsanity.com/paper/2206.08100/) (2022)
- [Semantic Communications in Networked Systems: A Data Significance Perspective](https://mlsanity.com/paper/2103.05391/) (2021)
- [Neuro-Symbolic Causal Reasoning Meets Signaling Game for Emergent Semantic Communications](https://mlsanity.com/paper/2210.12040/) (2022)
- [Semantic-Effectiveness Filtering and Control for Post-5G Wireless Connectivity](https://mlsanity.com/paper/1907.02441/) (2019)
- [Learning Semantics: An Opportunity for Effective 6G Communications](https://mlsanity.com/paper/2110.08049/) (2021)
- [Imitation Learning-based Implicit Semantic-aware Communication Networks: Multi-layer Representation and Collaborative Reasoning](https://mlsanity.com/paper/2210.16118/) (2022)
- [Curriculum Learning for Goal-Oriented Semantic Communications with a Common Language](https://mlsanity.com/paper/2204.10429/) (2022)
- [Causal Semantic Communication for Digital Twins: A Generalizable Imitation Learning Approach](https://mlsanity.com/paper/2304.12502/) (2023)
- [Emergence of Theory of Mind Collaboration in Multiagent Systems](https://mlsanity.com/paper/2110.00121/) (2021)
- [SNR-Independent Joint Source-Channel Coding for wireless image transmission](https://mlsanity.com/paper/2306.15183/) (2023)

## Built on by (most cited first, 5 of 5)

- [Artificial General Intelligence (AGI)-Native Wireless Systems: A Journey Beyond 6G](https://mlsanity.com/paper/2405.02336/) (2024)
- [Diffusion-Driven Semantic Communication for Generative Models with Bandwidth Constraints](https://mlsanity.com/paper/2407.18468/) (2024)
- [Dynamic Relative Representations for Goal-Oriented Semantic Communications](https://mlsanity.com/paper/2403.16986/) (2024)
- [Latent Space Alignment for AI-Native MIMO Semantic Communications](https://mlsanity.com/paper/2507.16680/) (2025)
- [Federated Latent Space Alignment for Multi-user Semantic Communications](https://mlsanity.com/paper/2602.17271/) (2026)

## Abstract (arXiv)

In this paper, a pragmatic semantic communication framework that enables effective goal-oriented information sharing between two-intelligent agents is proposed. In particular, semantics is defined as the causal state that encapsulates the fundamental causal relationships and dependencies among different features extracted from data. The proposed framework leverages the emerging concept in machine learning (ML) called theory of mind (ToM). It employs a dynamic two-level (wireless and semantic) feedback mechanism to continuously fine-tune neural network components at the transmitter. Thanks to the ToM, the transmitter mimics the actual mental state of the receiver's reasoning neural network operating semantic interpretation. Then, the estimated mental state at the receiver is dynamically updated thanks to the proposed dynamic two-level feedback mechanism. At the lower level, conventional channel quality metrics are used to optimize the channel encoding process based on the wireless communication channel's quality, ensuring an efficient mapping of semantic representations to a finite constellation. Additionally, a semantic feedback level is introduced, providing information on the receiver's perceived semantic effectiveness with minimal overhead. Numerical evaluations demonstrate the framework's ability to achieve efficient communication with a reduced amount of bits while maintaining the same semantics, outperforming conventional systems that do not exploit the ToM-based reasoning.