# Hierarchical Frequency-based Upsampling and Refining for Compressed Video Quality Enhancement

arXiv: https://arxiv.org/abs/2403.11556 · HTML: https://mlsanity.com/paper/2403.11556/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 50 references that are in the map.
- First posted on arXiv 18 Mar 2024.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution](https://mlsanity.com/paper/1704.03915/) (2017)
- [BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment](https://mlsanity.com/paper/2104.13371/) (2021)
- [Learning Enriched Features for Real Image Restoration and Enhancement](https://mlsanity.com/paper/2003.06792/) (2020)
- [MemNet: A Persistent Memory Network for Image Restoration](https://mlsanity.com/paper/1708.02209/) (2017)
- [Learning in the Frequency Domain](https://mlsanity.com/paper/2002.12416/) (2020)
- [FcaNet: Frequency Channel Attention Networks](https://mlsanity.com/paper/2012.11879/) (2020)
- [Compression Artifacts Reduction by a Deep Convolutional Network](https://mlsanity.com/paper/1504.06993/) (2015)
- [On the Frequency Bias of Generative Models](https://mlsanity.com/paper/2111.02447/) (2021)
- [Quantization Guided JPEG Artifact Correction](https://mlsanity.com/paper/2004.09320/) (2020)
- [Multi-Frame Quality Enhancement for Compressed Video](https://mlsanity.com/paper/1803.04680/) (2018)
- [Deep Generative Adversarial Compression Artifact Removal](https://mlsanity.com/paper/1704.02518/) (2017)
- [NTIRE 2021 Challenge on Quality Enhancement of Compressed Video: Methods and Results](https://mlsanity.com/paper/2104.10781/) (2021)
- [Learning Spatiotemporal Frequency-Transformer for Compressed Video Super-Resolution](https://mlsanity.com/paper/2208.03012/) (2022)
- [Feature Pyramid Network for Multi-Class Land Segmentation](https://mlsanity.com/paper/1806.03510/) (2018)
- [D3: Deep Dual-Domain Based Fast Restoration of JPEG-Compressed Images](https://mlsanity.com/paper/1601.04149/) (2016)
- [Implicit Dual-domain Convolutional Network for Robust Color Image Compression Artifact Reduction](https://mlsanity.com/paper/1810.08042/) (2018)
- [Learning Omni-frequency Region-adaptive Representations for Real Image Super-Resolution](https://mlsanity.com/paper/2012.06131/) (2020)
- [Recursive Fusion and Deformable Spatiotemporal Attention for Video Compression Artifact Reduction](https://mlsanity.com/paper/2108.02110/) (2021)
- [DPW-SDNet: Dual Pixel-Wavelet Domain Deep CNNs for Soft Decoding of JPEG-Compressed Images](https://mlsanity.com/paper/1805.10558/) (2018)
- [IPG-Net: Image Pyramid Guidance Network for Small Object Detection](https://mlsanity.com/paper/1912.00632/) (2019)
- [Transcoded Video Restoration by Temporal Spatial Auxiliary Network](https://mlsanity.com/paper/2112.07948/) (2021)

## Abstract (arXiv)

Video compression artifacts arise due to the quantization operation in the frequency domain. The goal of video quality enhancement is to reduce compression artifacts and reconstruct a visually-pleasant result. In this work, we propose a hierarchical frequency-based upsampling and refining neural network (HFUR) for compressed video quality enhancement. HFUR consists of two modules: implicit frequency upsampling module (ImpFreqUp) and hierarchical and iterative refinement module (HIR). ImpFreqUp exploits DCT-domain prior derived through implicit DCT transform, and accurately reconstructs the DCT-domain loss via a coarse-to-fine transfer. Consequently, HIR is introduced to facilitate cross-collaboration and information compensation between the scales, thus further refine the feature maps and promote the visual quality of the final output. We demonstrate the effectiveness of the proposed modules via ablation experiments and visualized results. Extensive experiments on public benchmarks show that HFUR achieves state-of-the-art performance for both constant bit rate and constant QP modes.