# GenStream: Semantic Streaming Framework for Generative Reconstruction of Human-centric Media

arXiv: https://arxiv.org/abs/2609.18634 · HTML: https://mlsanity.com/paper/2609.18634/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 60 references that are in the map.
- First posted on arXiv 16 Sep 2026.

## Builds on (most cited first, 24 of 24)

- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [TrackFormer: Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2101.02702/) (2021)
- [One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing](https://mlsanity.com/paper/2011.15126/) (2020)
- [Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fields](https://mlsanity.com/paper/2312.03203/) (2023)
- [DVC: An End-to-end Deep Video Compression Framework](https://mlsanity.com/paper/1812.00101/) (2018)
- [Softmax Splatting for Video Frame Interpolation](https://mlsanity.com/paper/2003.05534/) (2020)
- [NeRV: Neural Representations for Videos](https://mlsanity.com/paper/2110.13903/) (2021)
- [BlazePose: On-device Real-time Body Pose tracking](https://mlsanity.com/paper/2006.10204/) (2020)
- [SfM-Net: Learning of Structure and Motion from Video](https://mlsanity.com/paper/1704.07804/) (2017)
- [YOLO-Pose: Enhancing YOLO for Multi Person Pose Estimation Using Object Keypoint Similarity Loss](https://mlsanity.com/paper/2204.06806/) (2022)
- [Wireless Semantic Communications for Video Conferencing](https://mlsanity.com/paper/2204.07790/) (2022)
- [Deep Learning-Based Video Coding: A Review and A Case Study](https://mlsanity.com/paper/1904.12462/) (2019)
- [FAST: A Framework to Accelerate Super-Resolution Processing on Compressed Videos](https://mlsanity.com/paper/1603.08968/) (2016)
- [Implicit-explicit Integrated Representations for Multi-view Video Compression](https://mlsanity.com/paper/2311.17350/) (2023)
- [Recent Advances on HEVC Inter-frame Coding: From Optimization to Implementation and Beyond](https://mlsanity.com/paper/1910.09770/) (2019)
- [AI-Driven Innovations in Volumetric Video Streaming: A Review](https://mlsanity.com/paper/2412.12208/) (2024)
- [Adaptive Streaming in Interactive Multiview Video Systems](https://mlsanity.com/paper/1802.08994/) (2018)
- [Open GOP Resolution Switching in HTTP Adaptive Streaming with VVC](https://mlsanity.com/paper/2103.06675/) (2021)
- [DIGITWISE: Digital Twin-based Modeling of Adaptive Video Streaming Engagement](https://mlsanity.com/paper/2510.13267/) (2025)
- [End-to-End Learning-based Video Streaming Enhancement Pipeline: A Generative AI Approach](https://mlsanity.com/paper/2512.14185/) (2025)

## Abstract (arXiv)

Video streaming dominates global internet traffic, yet conventional pipelines remain inefficient for structured, human-centric content such as sports, performance, or interactive media. Standard codecs re-encode entire frames, foreground and background alike, treating all pixels uniformly and ignoring the semantic structure of the scene. This leads to significant bandwidth waste, particularly in scenarios where backgrounds are static and motion is constrained to a few salient actors. We introduce GenStream, a semantic streaming framework that replaces dense video frames with compact, structured metadata. Instead of transmitting pixels, GenStream encodes each scene as a combination of skeletal keypoints, camera viewpoint parameters, and a static 3D background model. These elements are transmitted to the client, where a generative model reconstructs photorealistic human figures and composites them into the 3D scene from the original viewpoint. This paradigm enables extreme compression, achieving over 99.9% bandwidth reduction compared to HEVC for the continuous data stream. We partially validate GenStream on Olympic figure skating footage and demonstrate potential for high perceptual fidelity under minimal data. While acknowledging the significant computational costs shifted to the client and challenges in generalization, GenStream opens new directions in volumetric avatar synthesis, canonical 3D actor fusion across views, and personalized viewing experiences, laying the groundwork for scalable, intelligent streaming in the post-codec era.