Embodied Semantic Communication for Collective Autonomous Agents: A Tutorial on Representation, Wireless Delivery, and Closed-Loop Coordination
Not on the map. Why?
Abstract (arXiv)
As autonomous systems and embodied intelligence enter the dynamic physical world, multi-agent collaboration calls for a paradigm shift in communication design. However, existing communication paradigms overlook that agents form action understanding from their own states, environmental observations, and collaboration relations through a process that evolves as a task unfolds. Consequently, reliable bit delivery, general semantic recovery, or single-task utility optimization alone cannot ensure that heterogeneous agents form coordinated actions compatible with their own conditions from shared information during task execution. To address this gap, this paper proposes embodied semantic communication (ESC) as a paradigm that transforms information transmission into action-oriented semantic interaction. Specifically, ESC characterizes how an explicit communication link can encapsulate multimodal perceptual states, intrinsic hardware capabilities, and collaborative intents into unified actionable semantic representations, thereby enabling heterogeneous receiving agents to parse, align, and ground them in local motor control. This paper clarifies the conceptual boundary, system characteristics, and environment-constrained technical pathways of ESC. It maps the underlying mathematical tools, including semantic information theory, world models, and multi-agent decision theory. Finally, this paper summarizes key open challenges, including measurable semantic reliability, ambiguity-triggered interaction under dynamic environments and tasks, and bandwidth-adaptive semantic transmission, outlining a roadmap for collective embodied networks.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 85 of its 138 references that are in the map. First posted on arXiv 28 Sep 2026.
Builds on
85 of 138 references.
References · 85
- 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
- 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
- 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
- 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
- 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
- 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
- 2022TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformerscited by 464
- 2022V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformercited by 320
- 2021OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communicationcited by 304
- 2022Learning robust perceptive locomotion for quadrupedal robots in the wildcited by 299
- 2022DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detectioncited by 270
- 2022CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformerscited by 229
38 more
- 2022CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformerscited by 227
- 2018TarMAC: Targeted Multi-Agent Communicationcited by 201
- 2023Multi-modal Learning with Missing Modality via Shared-Specific Feature Modellingcited by 159
- 2020When2com: Multi-Agent Perception via Communication Graph Groupingcited by 157
- 2024iVideoGPT: Interactive VideoGPTs are Scalable World Modelscited by 136
- 2019xMUDA: Cross-Modal Unsupervised Domain Adaptation for 3D Semantic Segmentationcited by 129
- 2022Robust Collaborative 3D Object Detection in Presence of Pose Errorscited by 129
- 2020Who2com: Collaborative Perception via Learnable Handshake Communicationcited by 127
- 2023Multimodal Prompting with Missing Modalities for Visual Recognitioncited by 121
- 2023From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thoughtcited by 118
- 2023V2X-Seq: A Large-Scale Sequential Dataset for Vehicle-Infrastructure Cooperative Perception and Forecastingcited by 114
- 20206G Networks: Beyond Shannon Towards Semantic and Goal-Oriented Communicationscited by 108
- 2022A Survey of Multi-Agent Deep Reinforcement Learning with Communicationcited by 103
- 2021Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weathercited by 95
- 2024IRASim: A Fine-Grained World Model for Robot Manipulationcited by 93
- 2021Semantic Communications: Principles and Challengescited by 90
- 2022Less Data, More Knowledge: Building Next Generation Semantic Communication Networkscited by 61
- 2022Uncertainty Quantification of Collaborative Detection for Self-Drivingcited by 46
- 2014Online Planning Algorithms for POMDPscited by 42
- 2024End-to-End Autonomous Driving through V2X Cooperationcited by 42
- 2021Planar Robot Casting with Real2Sim2Real Self-Supervised Learningcited by 38
- 2023Generative AI-driven Semantic Communication Networks: Architecture, Technologies and Applicationscited by 36
- 2023Heterogeneous Multi-Robot Reinforcement Learningcited by 34
- 2022CAT-Det: Contrastively Augmented Transformer for Multi-modal 3D Object Detectioncited by 32
- 2024L4DR: LiDAR-4DRadar Fusion for Weather-Robust 3D Object Detectioncited by 32
- 2021Deep Source-Channel Coding for Sentence Semantic Transmission with HARQcited by 29
- 2023UMC: A Unified Bandwidth-efficient and Multi-resolution based Collaborative Perception Frameworkcited by 29
- 2023Interruption-Aware Cooperative Perception for V2X Communication-Aided Autonomous Drivingcited by 25
- 2021Semantics-Native Communication with Contextual Reasoningcited by 21
- 2024Spatiotemporal Predictive Pre-training for Robotic Motor Controlcited by 21
- 2018A Survey of Knowledge Representation in Service Roboticscited by 18
- 2023M-BEV: Masked BEV Perception for Robust Autonomous Drivingcited by 18
- 2022A Critical Review of Communications in Multi-Robot Systemscited by 16
- 2025SparseAlign: A Fully Sparse Framework for Cooperative Object Detectioncited by 15
- 2024Generative AI for the Optimization of Next-Generation Wireless Networks: Basics, State-of-the-Art, and Open Challengescited by 14
- 2024MuDreamer: Learning Predictive World Models without Reconstructioncited by 13
- 2022Engineering Semantic Communication: A Surveycited by 11
- 2022Adaptive Bit Rate Control in Semantic Communication with Incremental Knowledge-based HARQcited by 10
50 most cited of 85 All 85 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Semantic Communication meets System 2 ML: How Abstraction, Compositionality and Emergent Languages Shape Intelligence
- 2026When Multi-Robot Systems Meet Agentic AI:Towards Embodied Collective Intelligence
- 2025Agentic AI-Empowered Conversational Embodied Intelligence Networks in 6G
- 2026Embodied Multi-Agent Coordination by Aligning World Models Through Dialogue
- 2026MECoBench: A Systematic Study of Multimodal Agent Collaboration in Embodied Environments
- 2025Communicating Plans, Not Percepts: Scalable Multi-Agent Coordination with Embodied World Models
- 2025Generative Multi-Agent Collaboration in Embodied AI: A Systematic Review
- 2026Hy-Embodied-VLM-1.0: Efficient Physical-World Agents