# Integrating Semantic Communication and Human Decision-Making into an End-to-End Sensing-Decision Framework

arXiv: https://arxiv.org/abs/2412.05103 · HTML: https://mlsanity.com/paper/2412.05103/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 99 references that are in the map.
- First posted on arXiv 6 Dec 2024.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [Deep Learning Enabled Semantic Communication Systems](https://mlsanity.com/paper/2006.10685/) (2020)
- [Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoff](https://mlsanity.com/paper/1901.07821/) (2019)
- [Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications](https://mlsanity.com/paper/2207.09353/) (2022)
- [6G Networks: Beyond Shannon Towards Semantic and Goal-Oriented Communications](https://mlsanity.com/paper/2011.14844/) (2020)
- [Learning Task-Oriented Communication for Edge Inference: An Information Bottleneck Approach](https://mlsanity.com/paper/2102.04170/) (2021)
- [Task-Oriented Multi-User Semantic Communications](https://mlsanity.com/paper/2112.10255/) (2021)
- [Deep Learning-Based Communication Over the Air](https://mlsanity.com/paper/1707.03384/) (2017)
- [Toward a 6G AI-Native Air Interface](https://mlsanity.com/paper/2012.08285/) (2020)
- [A Very Brief Introduction to Machine Learning With Applications to Communication Systems](https://mlsanity.com/paper/1808.02342/) (2018)
- [Large AI Model-Based Semantic Communications](https://mlsanity.com/paper/2307.03492/) (2023)
- [Deep Learning Enabled Semantic Communications with Speech Recognition and Synthesis](https://mlsanity.com/paper/2205.04603/) (2022)
- [Task-Oriented Sensing, Computation, and Communication Integration for Multi-Device Edge AI](https://mlsanity.com/paper/2207.00969/) (2022)
- [OFDM-Autoencoder for End-to-End Learning of Communications Systems](https://mlsanity.com/paper/1803.05815/) (2018)
- [LLMind: Orchestrating AI and IoT with LLM for Complex Task Execution](https://mlsanity.com/paper/2312.09007/) (2023)
- [A Brief Introduction to Machine Learning for Engineers](https://mlsanity.com/paper/1709.02840/) (2017)
- [Semantic Communications for Speech Signals](https://mlsanity.com/paper/2012.05369/) (2020)
- [Semantic Information Recovery in Wireless Networks](https://mlsanity.com/paper/2204.13366/) (2022)
- [Engineering Semantic Communication: A Survey](https://mlsanity.com/paper/2208.06314/) (2022)
- [Cognitive Semantic Communication Systems Driven by Knowledge Graph: Principle, Implementation, and Performance Evaluation](https://mlsanity.com/paper/2303.08546/) (2023)
- [Understand-Before-Talk (UBT): A Semantic Communication Approach to 6G Networks](https://mlsanity.com/paper/2210.06943/) (2022)
- [Semantic Communication for Cooperative Multi-Task Processing over Wireless Networks](https://mlsanity.com/paper/2404.08483/) (2024)
- [Cooperative and Collaborative Multi-Task Semantic Communication for Distributed Sources](https://mlsanity.com/paper/2411.02150/) (2024)
- [Model-free Reinforcement Learning of Semantic Communication by Stochastic Policy Gradient](https://mlsanity.com/paper/2305.03571/) (2023)
- [Rate-Distortion-Perception Theory for Semantic Communication](https://mlsanity.com/paper/2312.05437/) (2023)

## Abstract (arXiv)

As early as 1949, Weaver defined communication in a very broad sense to include all procedures by which one mind or technical system can influence another, thus establishing the idea of semantic communication. With the recent success of machine learning in expert assistance systems where sensed information is wirelessly provided to a human to assist task execution, the need to design effective and efficient communications has become increasingly apparent. In particular, semantic communication aims to convey the meaning behind the sensed information relevant for Human Decision-Making (HDM). Regarding the interplay between semantic communication and HDM, many questions remain, such as how to model the entire end-to-end sensing-decision-making process, how to design semantic communication for the HDM and which information should be provided for HDM. To address these questions, we propose to integrate semantic communication and HDM into one probabilistic end-to-end sensing-decision framework that bridges communications and psychology. In our interdisciplinary framework, we model the human through a HDM process, allowing us to explore how feature extraction from semantic communication can best support HDM both in theory and in simulations. In this sense, our study reveals the fundamental design trade-off between maximizing the relevant semantic information and matching the cognitive capabilities of the HDM model. Our initial analysis shows how semantic communication can balance the level of detail with human cognitive capabilities while demanding less bandwidth, power, and latency.