# DeformStream: Deformation-based Adaptive Volumetric Video Streaming

arXiv: https://arxiv.org/abs/2409.16615 · HTML: https://mlsanity.com/paper/2409.16615/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 40 references that are in the map.
- First posted on arXiv 25 Sep 2024.

## Builds on (most cited first, 7 of 7)

- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Video Based Reconstruction of 3D People Models](https://mlsanity.com/paper/1803.04758/) (2018)
- [How NeRFs and 3D Gaussian Splatting are Reshaping SLAM: a Survey](https://mlsanity.com/paper/2402.13255/) (2024)
- [Volumetric video streaming: Current approaches and implementations](https://mlsanity.com/paper/2209.01982/) (2022)
- [SLAMCast: Large-Scale, Real-Time 3D Reconstruction and Streaming for Immersive Multi-Client Live Telepresence](https://mlsanity.com/paper/1805.03709/) (2018)
- [LiveVV: Human-Centered Live Volumetric Video Streaming System](https://mlsanity.com/paper/2310.08205/) (2023)
- [Serverless Streaming for Emerging Media: Towards 5G Network-Driven Cost Optimization](https://mlsanity.com/paper/2102.04910/) (2021)

## Abstract (arXiv)

Volumetric video streaming offers immersive 3D experiences but faces significant challenges due to high bandwidth requirements and latency issues in transmitting detailed content in real time. Traditional methods like point cloud streaming compromise visual quality when zoomed in, and neural rendering techniques are too computationally intensive for real-time use. Though mesh-based streaming stands out by preserving surface detail and connectivity, offering a more refined representation for 3D content, traditional mesh streaming methods typically transmit data on a per-frame basis, failing to take full advantage of temporal redundancies across frames. This results in inefficient bandwidth usage and poor adaptability to fluctuating network conditions. We introduce Deformation-based Adaptive Volumetric Video Streaming, a novel framework that enhances volumetric video streaming performance by leveraging the inherent deformability of mesh-based representations. DeformStream uses embedded deformation to reconstruct subsequent frames from inter-frame motion, significantly reducing bandwidth usage while ensuring visual coherence between frames. To address frame reconstruction overhead and network adaptability, we formulate a new QoE model that accounts for client-side deformation latency and design a dynamic programming algorithm to optimize the trade-off between visual quality and bandwidth consumption under varying network conditions. Our evaluation demonstrates that Deformation-based Adaptive Volumetric Video Streaming outperforms existing mesh-based streaming systems in both bandwidth efficiency and visual quality, offering a robust solution for real-time volumetric video applications.