mlsanityData to 7 Oct 2026

Deep Neural Network-based Enhancement for Image and Video Streaming Systems: A Survey and Future Directions

arXiv 2106.03727 ↗First posted 7 Jun 2021eess.IV cs.CV cs.DCcited by 9

Not on the map. Why?

Abstract (arXiv)

Internet-enabled smartphones and ultra-wide displays are transforming a variety of visual apps spanning from on-demand movies and 360deg videos to video-conferencing and live streaming. However, robustly delivering visual content under fluctuating networking conditions on devices of diverse capabilities remains an open problem. In recent years, advances in the field of deep learning on tasks such as super-resolution and image enhancement have led to unprecedented performance in generating high-quality images from low-quality ones, a process we refer to as neural enhancement. In this paper, we survey state-of-the-art content delivery systems that employ neural enhancement as a key component in achieving both fast response time and high visual quality. We first present the components and architecture of existing content delivery systems, highlighting their challenges and motivating the use of neural enhancement models as a countermeasure. We then cover the deployment challenges of these models and analyze existing systems and their design decisions in efficiently overcoming these technical challenges. Additionally, we underline the key trends and common approaches across systems that target diverse use-cases. Finally, we present promising future directions based on the latest insights from deep learning research to further boost the quality of experience of content delivery systems.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 82 of its 180 references that are in the map. First posted on arXiv 7 Jun 2021.

Builds on

82 of 180 references.

References · 82

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  4. 2016Densely Connected Convolutional Networkscited by 6,691
  5. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  6. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  7. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  8. 2019Searching for MobileNetV3cited by 1,984
  9. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
  10. 2017Enhanced Deep Residual Networks for Single Image Super-Resolutioncited by 1,543
  11. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  12. 2017Deformable Convolutional Networkscited by 1,481
38 more
  1. 2015Spatial Transformer Networkscited by 1,423
  2. 2018Image Super-Resolution Using Very Deep Residual Channel Attention Networkscited by 1,256
  3. 2018ESRGAN: Enhanced Super-Resolution Generative Adversarial Networkscited by 1,250
  4. 2014Image Super-Resolution Using Deep Convolutional Networkscited by 1,218
  5. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,199
  6. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
  7. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
  8. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  9. 2015Accurate Image Super-Resolution Using Very Deep Convolutional Networkscited by 902
  10. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 780
  11. 2018Deformable ConvNets v2: More Deformable, Better Resultscited by 666
  12. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
  13. 2017The Perception-Distortion Tradeoffcited by 582
  14. 2019EDVR: Video Restoration with Enhanced Deformable Convolutional Networkscited by 492
  15. 2016Optical Flow Estimation using a Spatial Pyramid Networkcited by 421
  16. 2016Accelerating the Super-Resolution Convolutional Neural Networkcited by 393
  17. 2019Deep Learning for Image Super-resolution: A Surveycited by 368
  18. 2017SkipNet: Learning Dynamic Routing in Convolutional Networkscited by 328
  19. 2020BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyondcited by 303
  20. 2017"Zero-Shot" Super-Resolution using Deep Internal Learningcited by 285
  21. 2018Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Networkcited by 263
  22. 2017ICNet for Real-Time Semantic Segmentation on High-Resolution Imagescited by 261
  23. 2018TDAN: Temporally Deformable Alignment Network for Video Super-Resolutioncited by 225
  24. 2019Blind Super-Resolution With Iterative Kernel Correctioncited by 203
  25. 2019Blind Super-Resolution Kernel Estimation using an Internal-GANcited by 175
  26. 2018Frame-Recurrent Video Super-Resolutioncited by 160
  27. 2018Deep Learning for Single Image Super-Resolution: A Brief Reviewcited by 158
  28. 2020GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolutioncited by 158
  29. 2017Detail-revealing Deep Video Super-resolutioncited by 147
  30. 2018Dynamic Channel Pruning: Feature Boosting and Suppressioncited by 129
  31. 2018Fast and Accurate Single Image Super-Resolution via Information Distillation Networkcited by 121
  32. 2020Meta-Transfer Learning for Zero-Shot Super-Resolutioncited by 116
  33. 2020Structure-Preserving Super Resolution with Gradient Guidancecited by 116
  34. 2020Fast Bi-layer Neural Synthesis of One-Shot Realistic Head Avatarscited by 116
  35. 2020BRP-NAS: Prediction-based NAS using GCNscited by 109
  36. 2020SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloudcited by 109
  37. 2020Invertible Image Rescalingcited by 92
  38. 2020Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolutioncited by 91

50 most cited of 82 All 82 (JSON)

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 3; 2023: 1; 2024: 2; 2025: 1.

9 papers.

  1. 2023Dissecting Arbitrary-scale Super-resolution Capability from Pre-trained Diffusion Generative Modelscited by 15
  2. 2022fakeWeather: Adversarial Attacks for Deep Neural Networks Emulating Weather Conditions on the Camera Lens of Autonomous Systemscited by 8
  3. 2021OODIn: An Optimised On-Device Inference Framework for Heterogeneous Mobile Devicescited by 7
  4. 2024Zero-Shot Temporal Resolution Domain Adaptation for Spiking Neural Networkscited by 6
  5. 2021How to Reach Real-Time AI on Consumer Devices? Solutions for Programmable and Custom Architecturescited by 2
  6. 2022Meta-Learned Kernel For Blind Super-Resolution Kernel Estimationcited by 2
  7. 2022NAWQ-SR: A Hybrid-Precision NPU Engine for Efficient On-Device Super-Resolutioncited by 1
  8. 2025Real-Time Neural-Enhancement for Online Cloud Gamingcited by 1
  9. 2024Time-series Initialization and Conditioning for Video-agnostic Stabilization of Video Super-Resolution using Recurrent Networkscited by 0