# Toward Semantic Communication for Real-time Mobile 3D Reconstruction

arXiv: https://arxiv.org/abs/2607.16128 · HTML: https://mlsanity.com/paper/2607.16128/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 23 references that are in the map.
- First posted on arXiv 17 Jul 2026.

## Builds on (most cited first, 18 of 18)

- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [NICE-SLAM: Neural Implicit Scalable Encoding for SLAM](https://mlsanity.com/paper/2112.12130/) (2021)
- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [Neural RGB-D Surface Reconstruction](https://mlsanity.com/paper/2104.04532/) (2021)
- [NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video](https://mlsanity.com/paper/2104.00681/) (2021)
- [MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priors](https://mlsanity.com/paper/2412.12392/) (2024)
- [Co-SLAM: Joint Coordinate and Sparse Parametric Encodings for Neural Real-Time SLAM](https://mlsanity.com/paper/2304.14377/) (2023)
- [Photo-SLAM: Real-time Simultaneous Localization and Photorealistic Mapping for Monocular, Stereo, and RGB-D Cameras](https://mlsanity.com/paper/2311.16728/) (2023)
- [Robust Semantic Communications with Masked VQ-VAE Enabled Codebook](https://mlsanity.com/paper/2206.04011/) (2022)
- [A Unified Multi-Task Semantic Communication System for Multimodal Data](https://mlsanity.com/paper/2209.07689/) (2022)
- [Towards Semantic Communications: Deep Learning-Based Image Semantic Coding](https://mlsanity.com/paper/2208.04094/) (2022)
- [A Wireless AI-Generated Content (AIGC) Provisioning Framework Empowered by Semantic Communication](https://mlsanity.com/paper/2310.17705/) (2023)
- [S-RAN: Semantic-Aware Radio Access Networks](https://mlsanity.com/paper/2407.11161/) (2024)
- [Adaptive Semantic Communication for UAV/UGV Cooperative Path Planning](https://mlsanity.com/paper/2510.06901/) (2025)
- [Joint Knowledge and Power Management for Secure Semantic Communication Networks](https://mlsanity.com/paper/2504.15260/) (2025)
- [Integrated Sensing, Computing and Semantic Communication for Vehicular Networks](https://mlsanity.com/paper/2506.00522/) (2025)
- [Dynamic Control Aware Semantic Communication Enabled Image Transmission for Lunar Landing](https://mlsanity.com/paper/2510.06916/) (2025)

## Abstract (arXiv)

Real-time mobile 3D reconstruction is fundamental to many emerging applications such as autonomous navigation and digital twin construction, where a moving platform continuously captures an image stream and transmit to a computing server for scene understanding. Unlike offline reconstruction, camera poses and scene geometry are estimated on-the-fly during acquisition, making multi-view consistency a real-time requirement and rendering geometric estimation highly sensitive to communication-induced distortions. Semantic communication (SemCom) transmits compact semantic information, offering a promising way to preserve task-critical data over unreliable links. However, existing designs are optimized at the image or single-view level and without providing explicit reliability information for geometric estimation, limiting their applicability to real-time mobile 3D reconstruction. In this context, we propose a SemCom framework for real-time mobile 3D reconstruction. The framework includes a semantic transceiver that outputs a reconstructed image alongside a pixel-wise confidence map, quantifying the reliability of each region. We further introduce a confidence-guided geometric estimation method, incorporating confidence into RANSAC-based pose initialization and bundle adjustment to reduce the influence of unreliable regions and enhance robustness under noisy channels. Simulations show that, compared to existing SemCom and traditional seperate source and channel coding, our framework maintains high image quality while significantly improving pose estimation accuracy and 3D structural consistency.