# Fast-Vid2Vid: Spatial-Temporal Compression for Video-to-Video Synthesis

arXiv: https://arxiv.org/abs/2207.05049 · HTML: https://mlsanity.com/paper/2207.05049/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 53 references that are in the map.
- First posted on arXiv 11 Jul 2022.

## Builds on (most cited first, 30 of 42)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [First Order Motion Model for Image Animation](https://mlsanity.com/paper/2003.00196/) (2020)
- [Channel Pruning for Accelerating Very Deep Neural Networks](https://mlsanity.com/paper/1707.06168/) (2017)
- [Learning Efficient Convolutional Networks through Network Slimming](https://mlsanity.com/paper/1708.06519/) (2017)
- [DensePose: Dense Human Pose Estimation In The Wild](https://mlsanity.com/paper/1802.00434/) (2018)
- [End-to-End Video Instance Segmentation with Transformers](https://mlsanity.com/paper/2011.14503/) (2020)
- [Video-to-Video Synthesis](https://mlsanity.com/paper/1808.06601/) (2018)
- [Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks](https://mlsanity.com/paper/1808.06866/) (2018)
- [Everybody Dance Now](https://mlsanity.com/paper/1808.07371/) (2018)
- [Motion Representations for Articulated Animation](https://mlsanity.com/paper/2104.11280/) (2021)
- [Unifying distillation and privileged information](https://mlsanity.com/paper/1511.03643/) (2015)
- [Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures](https://mlsanity.com/paper/1607.03250/) (2016)
- [Few-shot Video-to-Video Synthesis](https://mlsanity.com/paper/1910.12713/) (2019)
- [Frame-Recurrent Video Super-Resolution](https://mlsanity.com/paper/1801.04590/) (2018)
- [A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers](https://mlsanity.com/paper/1804.03294/) (2018)
- [FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces](https://mlsanity.com/paper/1803.09179/) (2018)
- [Deep Flow-Guided Video Inpainting](https://mlsanity.com/paper/1905.02884/) (2019)
- [AutoGAN: Neural Architecture Search for Generative Adversarial Networks](https://mlsanity.com/paper/1908.03835/) (2019)
- [Anycost GANs for Interactive Image Synthesis and Editing](https://mlsanity.com/paper/2103.03243/) (2021)
- [Temporal Modulation Network for Controllable Space-Time Video Super-Resolution](https://mlsanity.com/paper/2104.10642/) (2021)
- [AutoGAN-Distiller: Searching to Compress Generative Adversarial Networks](https://mlsanity.com/paper/2006.08198/) (2020)

## Built on by (most cited first, 12 of 12)

- [Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models](https://mlsanity.com/paper/2402.17177/) (2024)
- [SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesis](https://mlsanity.com/paper/2303.16196/) (2023)
- [Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Control](https://mlsanity.com/paper/2503.14492/) (2025)
- [EffiVED:Efficient Video Editing via Text-instruction Diffusion Models](https://mlsanity.com/paper/2403.11568/) (2024)
- [From Efficient Multimodal Models to World Models: A Survey](https://mlsanity.com/paper/2407.00118/) (2024)
- [Traditional Classification Neural Networks are Good Generators: They are Competitive with DDPMs and GANs](https://mlsanity.com/paper/2211.14794/) (2022)
- [FastLightGen: Fast and Light Video Generation with Fewer Steps and Parameters](https://mlsanity.com/paper/2603.01685/) (2026)
- [NU-Class Net: A Novel Approach for Video Quality Enhancement](https://mlsanity.com/paper/2401.01163/) (2024)
- [Video to Video Generative Adversarial Network for Few-shot Learning Based on Policy Gradient](https://mlsanity.com/paper/2410.20657/) (2024)
- [Shortcut-V2V: Compression Framework for Video-to-Video Translation based on Temporal Redundancy Reduction](https://mlsanity.com/paper/2308.08011/) (2023)
- [SVS-GAN: Leveraging GANs for Semantic Video Synthesis](https://mlsanity.com/paper/2409.06074/) (2024)
- [VideoWeaver: Multimodal Multi-View Video-to-Video Transfer for Embodied Agents](https://mlsanity.com/paper/2603.25420/) (2026)

## Abstract (arXiv)

Video-to-Video synthesis (Vid2Vid) has achieved remarkable results in generating a photo-realistic video from a sequence of semantic maps. However, this pipeline suffers from high computational cost and long inference latency, which largely depends on two essential factors: 1) network architecture parameters, 2) sequential data stream. Recently, the parameters of image-based generative models have been significantly compressed via more efficient network architectures. Nevertheless, existing methods mainly focus on slimming network architectures and ignore the size of the sequential data stream. Moreover, due to the lack of temporal coherence, image-based compression is not sufficient for the compression of the video task. In this paper, we present a spatial-temporal compression framework, Fast-Vid2Vid, which focuses on data aspects of generative models. It makes the first attempt at time dimension to reduce computational resources and accelerate inference. Specifically, we compress the input data stream spatially and reduce the temporal redundancy. After the proposed spatial-temporal knowledge distillation, our model can synthesize key-frames using the low-resolution data stream. Finally, Fast-Vid2Vid interpolates intermediate frames by motion compensation with slight latency. On standard benchmarks, Fast-Vid2Vid achieves around real-time performance as 20 FPS and saves around 8x computational cost on a single V100 GPU.