mlsanityData to 14 Oct 2026

Real-World Video for Zoom Enhancement based on Spatio-Temporal Coupling

arXiv 2306.13875 ↗First posted 24 Jun 2023cs.CV eess.IVcited by 0

Not on the map. Why?

Abstract (arXiv)

In recent years, single-frame image super-resolution (SR) has become more realistic by considering the zooming effect and using real-world short- and long-focus image pairs. In this paper, we further investigate the feasibility of applying realistic multi-frame clips to enhance zoom quality via spatio-temporal information coupling. Specifically, we first built a real-world video benchmark, VideoRAW, by a synchronized co-axis optical system. The dataset contains paired short-focus raw and long-focus sRGB videos of different dynamic scenes. Based on VideoRAW, we then presented a Spatio-Temporal Coupling Loss, termed as STCL. The proposed STCL is intended for better utilization of information from paired and adjacent frames to align and fuse features both temporally and spatially at the feature level. The outperformed experimental results obtained in different zoom scenarios demonstrate the superiority of integrating real-world video dataset and STCL into existing SR models for zoom quality enhancement, and reveal that the proposed method can serve as an advanced and viable tool for video zoom.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 36 of its 50 references that are in the map. First posted on arXiv 24 Jun 2023.

Builds on

36 of 50 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  5. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
  6. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,908
  7. 2017Enhanced Deep Residual Networks for Single Image Super-Resolutioncited by 1,552
  8. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,436
  9. 2018Image Super-Resolution Using Very Deep Residual Channel Attention Networkscited by 1,264
  10. 2018ESRGAN: Enhanced Super-Resolution Generative Adversarial Networkscited by 1,254
  11. 2014Image Super-Resolution Using Deep Convolutional Networkscited by 1,224
  12. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,204
24 more
  1. 2015Accurate Image Super-Resolution Using Very Deep Convolutional Networkscited by 903
  2. 2017Video Enhancement with Task-Oriented Flowcited by 619
  3. 2019EDVR: Video Restoration with Enhanced Deformable Convolutional Networkscited by 494
  4. 2018Learning to See in the Darkcited by 460
  5. 2017Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolutioncited by 453
  6. 2019Toward Real-World Single Image Super-Resolution: A New Benchmark and A New Modelcited by 403
  7. 2018Scale-recurrent Network for Deep Image Deblurringcited by 333
  8. 2020Deep Unfolding Network for Image Super-Resolutioncited by 254
  9. 2018Frame-Recurrent Video Super-Resolutioncited by 160
  10. 2016Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensationcited by 156
  11. 2017Detail-revealing Deep Video Super-resolutioncited by 148
  12. 2018The Contextual Loss for Image Transformation with Non-Aligned Datacited by 135
  13. 2020Video Super-Resolution with Recurrent Structure-Detail Networkcited by 116
  14. 2019Zoom To Learn, Learn To Zoomcited by 115
  15. 2020Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolutioncited by 91
  16. 2019Handheld Multi-Frame Super-Resolutioncited by 85
  17. 2020Video Super-resolution with Temporal Group Attentioncited by 81
  18. 2019Camera Lens Super-Resolutioncited by 56
  19. 2018Multi-Frame Quality Enhancement for Compressed Videocited by 48
  20. 2020Deep Video Super-Resolution using HR Optical Flow Estimationcited by 40
  21. 2018Maintaining Natural Image Statistics with the Contextual Losscited by 19
  22. 2020ImagePairs: Realistic Super Resolution Dataset via Beam Splitter Camera Rigcited by 19
  23. 2017Fast and Accurate Image Super Resolution by Deep CNN with Skip Connection and Network in Networkcited by 17
  24. 2018Toward Bridging the Simulated-to-Real Gap: Benchmarking Super-Resolution on Real Datacited by 7

Built on by

Not cited yet.