# T-GVC: Trajectory-Guided Generative Video Coding at Ultra-Low Bitrates

arXiv: https://arxiv.org/abs/2507.07633 · HTML: https://mlsanity.com/paper/2507.07633/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 57 references that are in the map.
- First posted on arXiv 10 Jul 2025.

## Builds on (most cited first, 30 of 31)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [MotionCtrl: A Unified and Flexible Motion Controller for Video Generation](https://mlsanity.com/paper/2312.03641/) (2023)
- [VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models](https://mlsanity.com/paper/2401.09047/) (2024)
- [VideoCrafter1: Open Diffusion Models for High-Quality Video Generation](https://mlsanity.com/paper/2310.19512/) (2023)
- [CoTracker: It is Better to Track Together](https://mlsanity.com/paper/2307.07635/) (2023)
- [One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing](https://mlsanity.com/paper/2011.15126/) (2020)
- [A Survey on Video Diffusion Models](https://mlsanity.com/paper/2310.10647/) (2023)
- [Depth-Aware Generative Adversarial Network for Talking Head Video Generation](https://mlsanity.com/paper/2203.06605/) (2022)
- [Neural Video Compression with Diverse Contexts](https://mlsanity.com/paper/2302.14402/) (2023)
- [Tora: Trajectory-oriented Diffusion Transformer for Video Generation](https://mlsanity.com/paper/2407.21705/) (2024)
- [Neural Video Compression with Feature Modulation](https://mlsanity.com/paper/2402.17414/) (2024)
- [Motion-Conditioned Diffusion Model for Controllable Video Synthesis](https://mlsanity.com/paper/2304.14404/) (2023)
- [Improving Statistical Fidelity for Neural Image Compression with Implicit Local Likelihood Models](https://mlsanity.com/paper/2301.11189/) (2023)
- [Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent](https://mlsanity.com/paper/2411.02265/) (2024)
- [MOFA-Video: Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion Model](https://mlsanity.com/paper/2405.20222/) (2024)
- [Learning for Video Compression with Recurrent Auto-Encoder and Recurrent Probability Model](https://mlsanity.com/paper/2006.13560/) (2020)
- [Video Diffusion Models are Training-free Motion Interpreter and Controller](https://mlsanity.com/paper/2405.14864/) (2024)
- [Perceptual Learned Video Compression with Recurrent Conditional GAN](https://mlsanity.com/paper/2109.03082/) (2021)
- [Low Bandwidth Video-Chat Compression using Deep Generative Models](https://mlsanity.com/paper/2012.00328/) (2020)
- [ECVC: Exploiting Non-Local Correlations in Multiple Frames for Contextual Video Compression](https://mlsanity.com/paper/2410.09706/) (2024)
- [Generative Face Video Coding Techniques and Standardization Efforts: A Review](https://mlsanity.com/paper/2311.02649/) (2023)
- [Training-Free Motion-Guided Video Generation with Enhanced Temporal Consistency Using Motion Consistency Loss](https://mlsanity.com/paper/2501.07563/) (2025)
- [When Video Coding Meets Multimodal Large Language Models: A Unified Paradigm for Video Coding](https://mlsanity.com/paper/2408.08093/) (2024)
- [M3-CVC: Controllable Video Compression with Multimodal Generative Models](https://mlsanity.com/paper/2411.15798/) (2024)
- [Cross Modal Compression: Towards Human-comprehensible Semantic Compression](https://mlsanity.com/paper/2209.02574/) (2022)
- [Video Coding Using Learned Latent GAN Compression](https://mlsanity.com/paper/2207.04324/) (2022)

## Built on by (most cited first, 10 of 10)

- [Conditional Video Generation for High-Efficiency Video Compression](https://mlsanity.com/paper/2507.15269/) (2025)
- [Low-Bitrate Video Compression through Semantic-Conditioned Diffusion](https://mlsanity.com/paper/2512.00408/) (2025)
- [Dense-Prior-Guided Generative Video Compression](https://mlsanity.com/paper/2604.06655/) (2026)
- [ReGenVC: End-to-End Real-Time Generative Video Coding at Ultra-Low Bitrate](https://mlsanity.com/paper/2607.28144/) (2026)
- [A Causal Diffusion Model for Video Reconstruction from Ultra-Low-Bitrate Representations](https://mlsanity.com/paper/2602.13837/) (2026)
- [MTLSI-Net: A Linear Semantic Interaction Network for Parameter-Efficient Multi-Task Dense Prediction](https://mlsanity.com/paper/2604.01995/) (2026)
- [Group-of-Latents: Perceptual Video Compression at Extreme Bitrates via Masked Latent Generative Modeling](https://mlsanity.com/paper/2607.19437/) (2026)
- [GVCCTurbo: Rate-Compute Quality Scheduling for Codebook Driven Generative Compression](https://mlsanity.com/paper/2608.03517/) (2026)
- [Generative Video Compression Based on Hierarchical Referencing](https://mlsanity.com/paper/2608.11618/) (2026)
- [A Spatiotemporal Semantic Importance-Guided Unified Compression and Editing Framework for AI-Generated Videos](https://mlsanity.com/paper/2610.05779/) (2026)

## Abstract (arXiv)

Recent advances in video generation techniques have given rise to an emerging paradigm of generative video coding for Ultra-Low Bitrate (ULB) scenarios by leveraging powerful generative priors. However, most existing methods are limited by domain specificity (e.g., facial or human videos) or excessive dependence on high-level text guidance, which tend to inadequately capture fine-grained motion details, leading to unrealistic or incoherent reconstructions. To address these challenges, we propose Trajectory-Guided Generative Video Coding (dubbed T-GVC), a novel framework that bridges low-level motion tracking with high-level semantic understanding. T-GVC features a semantic-aware sparse motion sampling pipeline that extracts pixel-wise motion as sparse trajectory points based on their semantic importance, significantly reducing the bitrate while preserving critical temporal semantic information. In addition, by integrating trajectory-aligned loss constraints into diffusion processes, we introduce a training-free guidance mechanism in latent space to ensure physically plausible motion patterns without sacrificing the inherent capabilities of generative models. Experimental results demonstrate that T-GVC outperforms both traditional and neural video codecs under ULB conditions. Furthermore, additional experiments confirm that our framework achieves more precise motion control than existing text-guided methods, paving the way for a novel direction of generative video coding guided by geometric motion modeling.