# Transcoded Video Restoration by Temporal Spatial Auxiliary Network

arXiv: https://arxiv.org/abs/2112.07948 · HTML: https://mlsanity.com/paper/2112.07948/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 31 references that are in the map.
- First posted on arXiv 15 Dec 2021.

## Builds on (most cited first, 21 of 21)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [Optical Flow Estimation using a Spatial Pyramid Network](https://mlsanity.com/paper/1611.00850/) (2016)
- [Dilated Residual Networks](https://mlsanity.com/paper/1705.09914/) (2017)
- [TDAN: Temporally Deformable Alignment Network for Video Super-Resolution](https://mlsanity.com/paper/1812.02898/) (2018)
- [Compression Artifacts Reduction by a Deep Convolutional Network](https://mlsanity.com/paper/1504.06993/) (2015)
- [Understanding Deformable Alignment in Video Super-Resolution](https://mlsanity.com/paper/2009.07265/) (2020)
- [MFQE 2.0: A New Approach for Multi-frame Quality Enhancement on Compressed Video](https://mlsanity.com/paper/1902.09707/) (2019)
- [Multi-Frame Quality Enhancement for Compressed Video](https://mlsanity.com/paper/1803.04680/) (2018)
- [EfficientDeRain: Learning Pixel-wise Dilation Filtering for High-Efficiency Single-Image Deraining](https://mlsanity.com/paper/2009.09238/) (2020)
- [Enhancing Quality for HEVC Compressed Videos](https://mlsanity.com/paper/1709.06734/) (2017)
- [A Convolutional Neural Network Approach for Post-Processing in HEVC Intra Coding](https://mlsanity.com/paper/1608.06690/) (2016)
- [Deep Learning-Based Video Coding: A Review and A Case Study](https://mlsanity.com/paper/1904.12462/) (2019)
- [Enhancing HEVC Compressed Videos with a Partition-masked Convolutional Neural Network](https://mlsanity.com/paper/1805.03894/) (2018)

## Built on by (most cited first, 5 of 5)

- [CPGA: Coding Priors-Guided Aggregation Network for Compressed Video Quality Enhancement](https://mlsanity.com/paper/2403.10362/) (2024)
- [Beyond Alignment: Blind Video Face Restoration via Parsing-Guided Temporal-Coherent Transformer](https://mlsanity.com/paper/2404.13640/) (2024)
- [Hierarchical Frequency-based Upsampling and Refining for Compressed Video Quality Enhancement](https://mlsanity.com/paper/2403.11556/) (2024)
- [FCA2: Frame Compression-Aware Autoencoder for Modular and Fast Compressed Video Super-Resolution](https://mlsanity.com/paper/2506.11545/) (2025)
- [Blind Quality Enhancement of Compressed Video via Fine-Grained Degradation-Guided Sequential Inference](https://mlsanity.com/paper/2511.16137/) (2025)

## Abstract (arXiv)

In most video platforms, such as Youtube, and TikTok, the played videos usually have undergone multiple video encodings such as hardware encoding by recording devices, software encoding by video editing apps, and single/multiple video transcoding by video application servers. Previous works in compressed video restoration typically assume the compression artifacts are caused by one-time encoding. Thus, the derived solution usually does not work very well in practice. In this paper, we propose a new method, temporal spatial auxiliary network (TSAN), for transcoded video restoration. Our method considers the unique traits between video encoding and transcoding, and we consider the initial shallow encoded videos as the intermediate labels to assist the network to conduct self-supervised attention training. In addition, we employ adjacent multi-frame information and propose the temporal deformable alignment and pyramidal spatial fusion for transcoded video restoration. The experimental results demonstrate that the performance of the proposed method is superior to that of the previous techniques. The code is available at https://github.com/icecherylXuli/TSAN.