# A Global Appearance and Local Coding Distortion based Fusion Framework for CNN based Filtering in Video Coding

arXiv: https://arxiv.org/abs/2106.12746 · HTML: https://mlsanity.com/paper/2106.12746/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 61 references that are in the map.
- First posted on arXiv 24 Jun 2021.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Unprocessing Images for Learned Raw Denoising](https://mlsanity.com/paper/1811.11127/) (2018)
- [Residual Dense Network for Image Restoration](https://mlsanity.com/paper/1812.10477/) (2018)
- [Real Image Denoising with Feature Attention](https://mlsanity.com/paper/1904.07396/) (2019)
- [Compression Artifacts Reduction by a Deep Convolutional Network](https://mlsanity.com/paper/1504.06993/) (2015)
- [All you need is a good init](https://mlsanity.com/paper/1511.06422/) (2015)
- [Denoising Prior Driven Deep Neural Network for Image Restoration](https://mlsanity.com/paper/1801.06756/) (2018)
- [Deeply learned face representations are sparse, selective, and robust](https://mlsanity.com/paper/1412.1265/) (2014)
- [Hyperspectral Image Denoising Employing a Spatial-Spectral Deep Residual Convolutional Neural Network](https://mlsanity.com/paper/1806.00183/) (2018)
- [Multi-Frame Quality Enhancement for Compressed Video](https://mlsanity.com/paper/1803.04680/) (2018)
- [Connecting Image Denoising and High-Level Vision Tasks via Deep Learning](https://mlsanity.com/paper/1809.01826/) (2018)
- [CAS-CNN: A Deep Convolutional Neural Network for Image Compression Artifact Suppression](https://mlsanity.com/paper/1611.07233/) (2016)
- [Early Exit or Not: Resource-Efficient Blind Quality Enhancement for Compressed Images](https://mlsanity.com/paper/2006.16581/) (2020)
- [A Convolutional Neural Network Approach for Post-Processing in HEVC Intra Coding](https://mlsanity.com/paper/1608.06690/) (2016)
- [MFRNet: A New CNN Architecture for Post-Processing and In-loop Filtering](https://mlsanity.com/paper/2007.07099/) (2020)
- [PixelRL: Fully Convolutional Network with Reinforcement Learning for Image Processing](https://mlsanity.com/paper/1912.07190/) (2019)
- [A DenseNet Based Approach for Multi-Frame In-Loop Filter in HEVC](https://mlsanity.com/paper/1903.01648/) (2019)
- [Enhancing HEVC Compressed Videos with a Partition-masked Convolutional Neural Network](https://mlsanity.com/paper/1805.03894/) (2018)
- [Enhancing Quality for VVC Compressed Videos by Jointly Exploiting Spatial Details and Temporal Structure](https://mlsanity.com/paper/1901.09575/) (2019)
- [MGANet: A Robust Model for Quality Enhancement of Compressed Video](https://mlsanity.com/paper/1811.09150/) (2018)
- [A practical convolutional neural network as loop filter for intra frame](https://mlsanity.com/paper/1805.06121/) (2018)

## Built on by (most cited first, 2 of 2)

- [End-to-end Transformer for Compressed Video Quality Enhancement](https://mlsanity.com/paper/2210.13827/) (2022)
- [Compressed Video Quality Enhancement: Classifying and Benchmarking over Standards](https://mlsanity.com/paper/2509.10407/) (2025)

## Abstract (arXiv)

In-loop filtering is used in video coding to process the reconstructed frame in order to remove blocking artifacts. With the development of convolutional neural networks (CNNs), CNNs have been explored for in-loop filtering considering it can be treated as an image de-noising task. However, in addition to being a distorted image, the reconstructed frame is also obtained by a fixed line of block based encoding operations in video coding. It carries coding-unit based coding distortion of some similar characteristics. Therefore, in this paper, we address the filtering problem from two aspects, global appearance restoration for disrupted texture and local coding distortion restoration caused by fixed pipeline of coding. Accordingly, a three-stream global appearance and local coding distortion based fusion network is developed with a high-level global feature stream, a high-level local feature stream and a low-level local feature stream. Ablation study is conducted to validate the necessity of different features, demonstrating that the global features and local features can complement each other in filtering and achieve better performance when combined. To the best of our knowledge, we are the first one that clearly characterizes the video filtering process from the above global appearance and local coding distortion restoration aspects with experimental verification, providing a clear pathway to developing filter techniques. Experimental results demonstrate that the proposed method significantly outperforms the existing single-frame based methods and achieves 13.5%, 11.3%, 11.7% BD-Rate saving on average for AI, LDP and RA configurations, respectively, compared with the HEVC reference software.