# Neural Face Video Compression using Multiple Views

arXiv: https://arxiv.org/abs/2203.15401 · HTML: https://mlsanity.com/paper/2203.15401/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 27 references that are in the map.
- First posted on arXiv 29 Mar 2022.

## Builds on (most cited first, 20 of 20)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [First Order Motion Model for Image Animation](https://mlsanity.com/paper/2003.00196/) (2020)
- [One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing](https://mlsanity.com/paper/2011.15126/) (2020)
- [High-Fidelity Generative Image Compression](https://mlsanity.com/paper/2006.09965/) (2020)
- [DVC: An End-to-end Deep Video Compression Framework](https://mlsanity.com/paper/1812.00101/) (2018)
- [Few-Shot Adversarial Learning of Realistic Neural Talking Head Models](https://mlsanity.com/paper/1905.08233/) (2019)
- [Few-shot Video-to-Video Synthesis](https://mlsanity.com/paper/1910.12713/) (2019)
- [X2Face: A network for controlling face generation by using images, audio, and pose codes](https://mlsanity.com/paper/1807.10550/) (2018)
- [GANimation: Anatomically-aware Facial Animation from a Single Image](https://mlsanity.com/paper/1807.09251/) (2018)
- [Video Compression through Image Interpolation](https://mlsanity.com/paper/1804.06919/) (2018)
- [Fast Bi-layer Neural Synthesis of One-Shot Realistic Head Avatars](https://mlsanity.com/paper/2008.10174/) (2020)
- [Video Compression With Rate-Distortion Autoencoders](https://mlsanity.com/paper/1908.05717/) (2019)
- [MarioNETte: Few-shot Face Reenactment Preserving Identity of Unseen Targets](https://mlsanity.com/paper/1911.08139/) (2019)
- [ELF-VC: Efficient Learned Flexible-Rate Video Coding](https://mlsanity.com/paper/2104.14335/) (2021)
- [Low Bandwidth Video-Chat Compression using Deep Generative Models](https://mlsanity.com/paper/2012.00328/) (2020)
- [Ultra-low bitrate video conferencing using deep image animation](https://mlsanity.com/paper/2012.00346/) (2020)

## Built on by (most cited first, 11 of 11)

- [Generative Face Video Coding Techniques and Standardization Efforts: A Review](https://mlsanity.com/paper/2311.02649/) (2023)
- [Interactive Face Video Coding: A Generative Compression Framework](https://mlsanity.com/paper/2302.09919/) (2023)
- [Perceptual Quality Assessment of Face Video Compression: A Benchmark and An Effective Method](https://mlsanity.com/paper/2304.07056/) (2023)
- [Generative Visual Compression: A Review](https://mlsanity.com/paper/2402.02140/) (2024)
- [Gemino: Practical and Robust Neural Compression for Video Conferencing](https://mlsanity.com/paper/2209.10507/) (2022)
- [Generative Human Video Compression with Multi-granularity Temporal Trajectory Factorization](https://mlsanity.com/paper/2410.10171/) (2024)
- [Tokenizing Motion: A Generative Approach for Scene Dynamics Compression](https://mlsanity.com/paper/2410.09768/) (2024)
- [Pleno-Generation: A Scalable Generative Face Video Compression Framework with Bandwidth Intelligence](https://mlsanity.com/paper/2502.17085/) (2025)
- [Generative Models at the Frontier of Compression: A Survey on Generative Face Video Coding](https://mlsanity.com/paper/2506.07369/) (2025)
- [Resolution-Agnostic Neural Compression for High-Fidelity Portrait Video Conferencing via Implicit Radiance Fields](https://mlsanity.com/paper/2402.16599/) (2024)
- [One-Click Upgrade from 2D to 3D: Sandwiched RGB-D Video Compression for Stereoscopic Teleconferencing](https://mlsanity.com/paper/2404.09979/) (2024)

## Abstract (arXiv)

Recent advances in deep generative models led to the development of neural face video compression codecs that use an order of magnitude less bandwidth than engineered codecs. These neural codecs reconstruct the current frame by warping a source frame and using a generative model to compensate for imperfections in the warped source frame. Thereby, the warp is encoded and transmitted using a small number of keypoints rather than a dense flow field, which leads to massive savings compared to traditional codecs. However, by relying on a single source frame only, these methods lead to inaccurate reconstructions (e.g. one side of the head becomes unoccluded when turning the head and has to be synthesized). Here, we aim to tackle this issue by relying on multiple source frames (views of the face) and present encouraging results.