# Task-Oriented Communications for 3D Scene Representation: Balancing Timeliness and Fidelity

arXiv: https://arxiv.org/abs/2509.17282 · HTML: https://mlsanity.com/paper/2509.17282/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 36 of its 59 references that are in the map.
- First posted on arXiv 21 Sep 2025.

## Builds on (most cited first, 30 of 36)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [K-Planes: Explicit Radiance Fields in Space, Time, and Appearance](https://mlsanity.com/paper/2301.10241/) (2023)
- [Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans](https://mlsanity.com/paper/2012.15838/) (2020)
- [Nerfstudio: A Modular Framework for Neural Radiance Field Development](https://mlsanity.com/paper/2302.04264/) (2023)
- [NICE-SLAM: Neural Implicit Scalable Encoding for SLAM](https://mlsanity.com/paper/2112.12130/) (2021)
- [iMAP: Implicit Mapping and Positioning in Real-Time](https://mlsanity.com/paper/2103.12352/) (2021)
- [MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction](https://mlsanity.com/paper/2206.00665/) (2022)
- [Advances in Neural Rendering](https://mlsanity.com/paper/2111.05849/) (2021)
- [Octree-GS: Towards Consistent Real-time Rendering with LOD-Structured 3D Gaussians](https://mlsanity.com/paper/2403.17898/) (2024)
- [NeRF-SLAM: Real-Time Dense Monocular SLAM with Neural Radiance Fields](https://mlsanity.com/paper/2210.13641/) (2022)
- [Learning Task-Oriented Communication for Edge Inference: An Information Bottleneck Approach](https://mlsanity.com/paper/2102.04170/) (2021)
- [VR-NeRF: High-Fidelity Virtualized Walkable Spaces](https://mlsanity.com/paper/2311.02542/) (2023)
- [Towards Practical Real-Time Neural Video Compression](https://mlsanity.com/paper/2502.20762/) (2025)
- [KV Cache Compression, But What Must We Give in Return? A Comprehensive Benchmark of Long Context Capable Approaches](https://mlsanity.com/paper/2407.01527/) (2024)
- [Instant Gaussian Stream: Fast and Generalizable Streaming of Dynamic Scene Reconstruction via Gaussian Splatting](https://mlsanity.com/paper/2503.16979/) (2025)
- [Personalized Saliency in Task-Oriented Semantic Communications: Image Transmission and Performance Analysis](https://mlsanity.com/paper/2209.12274/) (2022)
- [Unsupervised Learning of Depth, Optical Flow and Pose with Occlusion from 3D Geometry](https://mlsanity.com/paper/2003.00766/) (2020)
- [Computer Vision Aided mmWave Beam Alignment in V2X Communications](https://mlsanity.com/paper/2207.11409/) (2022)
- [Single-step deep reinforcement learning for open-loop control of laminar and turbulent flows](https://mlsanity.com/paper/2006.02979/) (2020)
- [Neural radiance fields in the industrial and robotics domain: applications, research opportunities and use cases](https://mlsanity.com/paper/2308.07118/) (2023)
- [NerfBridge: Bringing Real-time, Online Neural Radiance Field Training to Robotics](https://mlsanity.com/paper/2305.09761/) (2023)
- [Task-Oriented Cross-System Design for Timely and Accurate Modeling in the Metaverse](https://mlsanity.com/paper/2309.05622/) (2023)
- [Progressively Deformable 2D Gaussian Splatting for Video Representation at Arbitrary Resolutions](https://mlsanity.com/paper/2503.05600/) (2025)
- [Task-Oriented Metaverse Design in the 6G Era](https://mlsanity.com/paper/2306.03158/) (2023)
- [Effective Communication: When to Pull Updates?](https://mlsanity.com/paper/2311.06432/) (2023)

## Abstract (arXiv)

Real-time Three-dimensional (3D) scene representation is a foundational element that supports a broad spectrum of cutting-edge applications, including digital manufacturing, Virtual, Augmented, and Mixed Reality (VR/AR/MR), and the emerging metaverse. Despite advancements in real-time communication and computing, achieving a balance between timeliness and fidelity in 3D scene representation remains a challenge. This work investigates a wireless network where multiple homogeneous mobile robots, equipped with cameras, capture an environment and transmit images to an edge server over channels for 3D representation. We propose a contextual-bandit Proximal Policy Optimization (PPO) framework incorporating both Age of Information (AoI) and semantic information to optimize image selection for representation, balancing data freshness and representation quality. Two policies -- the ω-threshold and ω-wait policies -- together with two benchmark methods are evaluated, timeliness embedding and weighted sum, on standard datasets and baseline 3D scene representation models. Experimental results demonstrate improved representation fidelity while maintaining low latency, offering insight into the model's decision-making process. This work advances real-time 3D scene representation by optimizing the trade-off between timeliness and fidelity in dynamic environments.