# Gemino: Practical and Robust Neural Compression for Video Conferencing

arXiv: https://arxiv.org/abs/2209.10507 · HTML: https://mlsanity.com/paper/2209.10507/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 71 references that are in the map.
- First posted on arXiv 21 Sep 2022.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [First Order Motion Model for Image Animation](https://mlsanity.com/paper/2003.00196/) (2020)
- [One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing](https://mlsanity.com/paper/2011.15126/) (2020)
- [Video-to-Video Synthesis](https://mlsanity.com/paper/1808.06601/) (2018)
- [VRT: A Video Restoration Transformer](https://mlsanity.com/paper/2201.12288/) (2022)
- [FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors](https://mlsanity.com/paper/1711.10703/) (2017)
- [Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation](https://mlsanity.com/paper/1611.05250/) (2016)
- [NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applications](https://mlsanity.com/paper/1804.03230/) (2018)
- [Fast Bi-layer Neural Synthesis of One-Shot Realistic Head Avatars](https://mlsanity.com/paper/2008.10174/) (2020)
- [FastDepth: Fast Monocular Depth Estimation on Embedded Systems](https://mlsanity.com/paper/1903.03273/) (2019)
- [Deep Face Super-Resolution with Iterative Collaboration between Attentive Recovery and Landmark Estimation](https://mlsanity.com/paper/2003.13063/) (2020)
- [PyTorch Image Quality: Metrics for Image Quality Assessment](https://mlsanity.com/paper/2208.14818/) (2022)
- [Low Bandwidth Video-Chat Compression using Deep Generative Models](https://mlsanity.com/paper/2012.00328/) (2020)
- [Efficient Video Compression via Content-Adaptive Super-Resolution](https://mlsanity.com/paper/2104.02322/) (2021)
- [Neural Face Video Compression using Multiple Views](https://mlsanity.com/paper/2203.15401/) (2022)
- [FAST: A Framework to Accelerate Super-Resolution Processing on Compressed Videos](https://mlsanity.com/paper/1603.08968/) (2016)

## Built on by (most cited first, 8 of 8)

- [Glia: A Human-Inspired AI for Automated Systems Design and Optimization](https://mlsanity.com/paper/2510.27176/) (2025)
- [GRACE: Loss-Resilient Real-Time Video through Neural Codecs](https://mlsanity.com/paper/2305.12333/) (2023)
- [Reparo: Loss-Resilient Generative Codec for Video Conferencing](https://mlsanity.com/paper/2305.14135/) (2023)
- [AVA: Towards Agentic Video Analytics with Vision Language Models](https://mlsanity.com/paper/2505.00254/) (2025)
- [Promptus: Can Prompts Streaming Replace Video Streaming with Stable Diffusion](https://mlsanity.com/paper/2405.20032/) (2024)
- [Generative Models at the Frontier of Compression: A Survey on Generative Face Video Coding](https://mlsanity.com/paper/2506.07369/) (2025)
- [Responsive DNN Adaptation for Video Analytics against Environment Shift via Hierarchical Mobile-Cloud Collaborations](https://mlsanity.com/paper/2505.00745/) (2025)
- [A Camera-Native Talking-Head Video Dataset for Various Computer Vision Tasks](https://mlsanity.com/paper/2603.26763/) (2026)

## Abstract (arXiv)

Video conferencing systems suffer from poor user experience when network conditions deteriorate because current video codecs simply cannot operate at extremely low bitrates. Recently, several neural alternatives have been proposed that reconstruct talking head videos at very low bitrates using sparse representations of each frame such as facial landmark information. However, these approaches produce poor reconstructions in scenarios with major movement or occlusions over the course of a call, and do not scale to higher resolutions. We design Gemino, a new neural compression system for video conferencing based on a novel high-frequency-conditional super-resolution pipeline. Gemino upsamples a very low-resolution version of each target frame while enhancing high-frequency details (e.g., skin texture, hair, etc.) based on information extracted from a single high-resolution reference image. We use a multi-scale architecture that runs different components of the model at different resolutions, allowing it to scale to resolutions comparable to 720p, and we personalize the model to learn specific details of each person, achieving much better fidelity at low bitrates. We implement Gemino atop aiortc, an open-source Python implementation of WebRTC, and show that it operates on 1024x1024 videos in real-time on a Titan X GPU, and achieves 2.2-5x lower bitrate than traditional video codecs for the same perceptual quality.