Perceptual Refinement of an End-to-End Video Streaming Pipeline via Generative AI Layers
Not on the map. Why?
Abstract (arXiv)
Traditional codecs treat every region of a frame alike; a generative layer can instead degrade the regions a viewer attends to least and reconstruct them at the client. We present PRESLEY, which extends the prior conference work ELVIS by replacing destructive block removal with adaptive in-place degradation under a removability mask, signaling per-block strength in a bit-packed side channel, and restoring via generative backbones conditioned on transmitted visual priors rather than unconditioned in-painting. We separate the problem into three goals: choosing which blocks to degrade, degrading them so the encoder spends fewer bits, and restoring them. Against its predecessor at matched rate, PRESLEY achieves a decisive mean -56.4% BD-rate reduction on delivered background quality across 13 rate ladders spanning multiple codecs and dataset families. Against pristine baselines, PRESLEY defines the operating regime of generative transport: delivering substantial bitrate savings (up to -29.4% BD-rate) and superior background quality (17/23 sequences) in the target bit-starved regime, while maintaining foreground fidelity bit-exact. We further map where the theoretical headroom in this class of architecture lies. Using an exact leave-one-superblock-out combinatorial oracle as an additive empirical bound, we show that existing complexity heuristics already capture 83.3% of bit-cost savings, bounding remaining cost-axis headroom at about 5% of total bitrate. We then identify and model the primary unaddressed axis -- post-restoration damage -- which disperses widely (4.9-8.4 dB). We prove that this damage is predictable before transmission (held-out rho = +0.400), establishing the feasibility of transmit-time restorability modeling and defining the roadmap for joint rate-distortion-restoration selection rules.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 53 references that are in the map. First posted on arXiv 16 Sep 2026.
Builds on
22 of 53 references.
References · 22
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2018ESRGAN: Enhanced Super-Resolution Generative Adversarial Networkscited by 1,254
- 2021Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Datacited by 970
- 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 957
- 2020Image Quality Assessment: Unifying Structure and Texture Similaritycited by 811
- 2021Designing a Practical Degradation Model for Deep Blind Image Super-Resolutioncited by 529
- 2022Activating More Pixels in Image Super-Resolution Transformercited by 438
- 2018YouTube-VOS: A Large-Scale Video Object Segmentation Benchmarkcited by 383
- 2019Depth-Aware Video Frame Interpolationcited by 234
- 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenescited by 234
- 2017Video Frame Interpolation via Adaptive Separable Convolutioncited by 213
- 2023ProPainter: Improving Propagation and Transformer for Video Inpaintingcited by 197
10 more
- 2020Learning Joint Spatial-Temporal Transformations for Video Inpaintingcited by 163
- 2021Video Super-Resolution Transformercited by 121
- 2022Towards An End-to-End Framework for Flow-Guided Video Inpaintingcited by 113
- 2024Fr'echet Video Motion Distance: A Metric for Evaluating Motion Consistency in Videoscited by 58
- 2020A Technical Overview of AV1cited by 47
- 2015Video Inpainting of Complex Scenescited by 41
- 2019FISR: Deep Joint Frame Interpolation and Super-Resolution with a Multi-scale Temporal Losscited by 28
- 2023Flow-Guided Diffusion for Video Inpaintingcited by 11
- 2025Stream-DiffVSR: Low-Latency Streamable Video Super-Resolution via Auto-Regressive Diffusioncited by 7
- 2024InstantIR: Blind Image Restoration with Instant Generative Referencecited by 3
Built on by
Not cited yet.