# A Semantic Communication System for Real-time 3D Reconstruction Tasks

arXiv: https://arxiv.org/abs/2412.01191 · HTML: https://mlsanity.com/paper/2412.01191/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 16 references that are in the map.
- First posted on arXiv 2 Dec 2024.

## Builds on (most cited first, 9 of 9)

- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [Deep Learning Enabled Semantic Communication Systems](https://mlsanity.com/paper/2006.10685/) (2020)
- [Semantic Communications for Future Internet: Fundamentals, Applications, and Challenges](https://mlsanity.com/paper/2207.00427/) (2022)
- [Wireless Semantic Communications for Video Conferencing](https://mlsanity.com/paper/2204.07790/) (2022)
- [SCTNet: Single-Branch CNN with Transformer Semantic Information for Real-Time Segmentation](https://mlsanity.com/paper/2312.17071/) (2023)
- [A Robust Deep Learning Enabled Semantic Communication System for Text](https://mlsanity.com/paper/2206.02596/) (2022)
- [Demo: Real-Time Semantic Communications with a Vision Transformer](https://mlsanity.com/paper/2205.03886/) (2022)
- [Automatic and Flexible Transmission of Semantic Map Images using Polar Codes for End-to-End Semantic-based Communication Systems](https://mlsanity.com/paper/2304.13632/) (2023)

## Built on by (most cited first, 2 of 2)

- [Channel-adaptive Cross-modal Generative Semantic Communication for Point Cloud Transmission](https://mlsanity.com/paper/2506.03211/) (2025)
- [Task-Oriented Communications for 3D Scene Representation: Balancing Timeliness and Fidelity](https://mlsanity.com/paper/2509.17282/) (2025)

## Abstract (arXiv)

3D semantic maps have played an increasingly important role in high-precision robot localization and scene understanding. However, real-time construction of semantic maps requires mobile edge devices with extremely high computing power, which are expensive and limit the widespread application of semantic mapping. In order to address this limitation, inspired by cloud-edge collaborative computing and the high transmission efficiency of semantic communication, this paper proposes a method to achieve real-time semantic mapping tasks with limited-resource mobile devices. Specifically, we design an encoding-decoding semantic communication framework for real-time semantic mapping tasks under limited-resource situations. In addition, considering the impact of different channel conditions on communication, this paper designs a module based on the attention mechanism to achieve stable data transmission under various channel conditions. In terms of simulation experiments, based on the TUM dataset, it was verified that the system has an error of less than 0.1% compared to the groundtruth in mapping and localization accuracy and is superior to some novel semantic communication algorithms in real-time performance and channel adaptation. Besides, we implement a prototype system to verify the effectiveness of the proposed framework and designed module in real indoor scenarios. The results show that our system can complete real-time semantic mapping tasks for common indoor objects (chairs, computers, people, etc.) with a limited-resource device, and the mapping update time is less than 1 second.