# Low-Bitrate Video Compression through Semantic-Conditioned Diffusion

arXiv: https://arxiv.org/abs/2512.00408 · HTML: https://mlsanity.com/paper/2512.00408/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 62 references that are in the map.
- First posted on arXiv 29 Nov 2025.

## Builds on (most cited first, 30 of 43)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [Image Quality Assessment: Unifying Structure and Texture Similarity](https://mlsanity.com/paper/2004.07728/) (2020)
- [Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules](https://mlsanity.com/paper/2001.01568/) (2020)
- [ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting](https://mlsanity.com/paper/2307.12348/) (2023)
- [Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectives](https://mlsanity.com/paper/2211.04894/) (2022)
- [Effective Whole-body Pose Estimation with Two-stages Distillation](https://mlsanity.com/paper/2307.15880/) (2023)
- [NeRV: Neural Representations for Videos](https://mlsanity.com/paper/2110.13903/) (2021)
- [Neural Video Compression with Feature Modulation](https://mlsanity.com/paper/2402.17414/) (2024)
- [LDMVFI: Video Frame Interpolation with Latent Diffusion Models](https://mlsanity.com/paper/2303.09508/) (2023)
- [Learning to generate line drawings that convey geometry and semantics](https://mlsanity.com/paper/2203.12691/) (2022)
- [Towards Practical Real-Time Neural Video Compression](https://mlsanity.com/paper/2502.20762/) (2025)
- [OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generation](https://mlsanity.com/paper/2412.00115/) (2024)
- [Fr'echet Video Motion Distance: A Metric for Evaluating Motion Consistency in Videos](https://mlsanity.com/paper/2407.16124/) (2024)
- [C3: High-performance and low-complexity neural compression from a single image or video](https://mlsanity.com/paper/2312.02753/) (2023)
- [UniRestore: Unified Perceptual and Task-Oriented Image Restoration Model Using Diffusion Prior](https://mlsanity.com/paper/2501.13134/) (2025)
- [Perceptual Learned Video Compression with Recurrent Conditional GAN](https://mlsanity.com/paper/2109.03082/) (2021)
- [FloLPIPS: A Bespoke Video Quality Metric for Frame Interpoation](https://mlsanity.com/paper/2207.08119/) (2022)
- [MISC: Ultra-low Bitrate Image Semantic Compression Driven by Large Multimodal Model](https://mlsanity.com/paper/2402.16749/) (2024)
- [NVRC: Neural Video Representation Compression](https://mlsanity.com/paper/2409.07414/) (2024)
- [Generative Latent Coding for Ultra-Low Bitrate Image and Video Compression](https://mlsanity.com/paper/2505.16177/) (2025)
- [Diffusion-Driven Semantic Communication for Generative Models with Bandwidth Constraints](https://mlsanity.com/paper/2407.18468/) (2024)
- [EDEN: Enhanced Diffusion for High-quality Large-motion Video Frame Interpolation](https://mlsanity.com/paper/2503.15831/) (2025)
- [UltraHR-100K: Enhancing UHR Image Synthesis with A Large-Scale High-Quality Dataset](https://mlsanity.com/paper/2510.20661/) (2025)
- [Semantics-Guided Diffusion for Deep Joint Source-Channel Coding in Wireless Image Transmission](https://mlsanity.com/paper/2501.01138/) (2025)

## Built on by (most cited first, 4 of 4)

- [ReGenVC: End-to-End Real-Time Generative Video Coding at Ultra-Low Bitrate](https://mlsanity.com/paper/2607.28144/) (2026)
- [NeuralLVC: Neural Lossless Video Compression via Masked Diffusion with Temporal Conditioning](https://mlsanity.com/paper/2604.03353/) (2026)
- [GVCCTurbo: Rate-Compute Quality Scheduling for Codebook Driven Generative Compression](https://mlsanity.com/paper/2608.03517/) (2026)
- [A Spatiotemporal Semantic Importance-Guided Unified Compression and Editing Framework for AI-Generated Videos](https://mlsanity.com/paper/2610.05779/) (2026)

## Abstract (arXiv)

Traditional video codecs optimized for pixel fidelity collapse at ultra-low bitrates and produce severe artifacts. This failure arises from a fundamental misalignment between pixel accuracy and human perception. We propose a semantic video compression framework named DiSCo that transmits only the most meaningful information while relying on generative priors for detail synthesis. The source video is decomposed into three compact modalities: a textual description, a spatiotemporally degraded video, and optional sketches or poses that respectively capture semantic, appearance, and motion cues. A conditional video diffusion model then reconstructs high-quality, temporally coherent videos from these compact representations. Temporal forward filling, token interleaving, and modality-specific codecs are proposed to improve multimodal generation and modality compactness. Experiments show that our method outperforms baseline semantic and traditional codecs by 2-10X on perceptual metrics at low bitrates.