# Video Compression with Arbitrary Rescaling Network

arXiv: https://arxiv.org/abs/2306.04202 · HTML: https://mlsanity.com/paper/2306.04202/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 12 references that are in the map.
- First posted on arXiv 7 Jun 2023.

## Builds on (most cited first, 8 of 8)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [Enhanced Standard Compatible Image Compression Framework based on Auxiliary Codec Networks](https://mlsanity.com/paper/2009.14754/) (2020)
- [RR-DnCNN v2.0: Enhanced Restoration-Reconstruction Deep Neural Network for Down-Sampling Based Video Coding](https://mlsanity.com/paper/2002.10739/) (2020)
- [Deep Video Precoding](https://mlsanity.com/paper/1908.00812/) (2019)

## Built on by (most cited first, 2 of 2)

- [SCALED : Surrogate-gradient for Codec-Aware Learning of Downsampling in ABR Streaming](https://mlsanity.com/paper/2602.00198/) (2026)
- [A Projection-Based Surrogate Gradient Interpretation for Neural Codec Wrappers](https://mlsanity.com/paper/2606.20671/) (2026)

## Abstract (arXiv)

Most video platforms provide video streaming services with different qualities, and the quality of the services is usually adjusted by the resolution of the videos. So high-resolution videos need to be downsampled for compression. In order to solve the problem of video coding at different resolutions, we propose a rate-guided arbitrary rescaling network (RARN) for video resizing before encoding. To help the RARN be compatible with standard codecs and generate compression-friendly results, an iteratively optimized transformer-based virtual codec (TVC) is introduced to simulate the key components of video encoding and perform bitrate estimation. By iteratively training the TVC and the RARN, we achieved 5%-29% BD-Rate reduction anchored by linear interpolation under different encoding configurations and resolutions, exceeding the previous methods on most test videos. Furthermore, the lightweight RARN structure can process FHD (1080p) content at real-time speed (91 FPS) and obtain a considerable rate reduction.