# Free-VSC: Free Semantics from Visual Foundation Models for Unsupervised Video Semantic Compression

arXiv: https://arxiv.org/abs/2409.11718 · HTML: https://mlsanity.com/paper/2409.11718/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Sep 2024.

## Built on by (most cited first, 10 of 10)

- [Prompt-based Adaptation in Large-scale Vision Models: A Survey](https://mlsanity.com/paper/2510.13219/) (2025)
- [Towards All-in-One Medical Image Re-Identification](https://mlsanity.com/paper/2503.08173/) (2025)
- [DT-UFC: Universal Large Model Feature Coding via Peaky-to-Balanced Distribution Transformation](https://mlsanity.com/paper/2506.16495/) (2025)
- [Medical Manifestation-Aware De-Identification](https://mlsanity.com/paper/2412.10804/) (2024)
- [ProGVC: Progressive-based Generative Video Compression via Auto-Regressive Context Modeling](https://mlsanity.com/paper/2603.17546/) (2026)
- [Semantics versus Identity: A Divide-and-Conquer Approach towards Adjustable Medical Image De-Identification](https://mlsanity.com/paper/2507.21703/) (2025)
- [The Describe-Then-Generate Bottleneck: How VLM Descriptions Alter Image Generation Outcomes](https://mlsanity.com/paper/2509.18179/) (2025)
- [ROFI: A Deep Learning-Based Ophthalmic Sign-Preserving and Reversible Patient Face Anonymizer](https://mlsanity.com/paper/2510.11073/) (2025)
- [KD-NVC: A Search-and-Distill Framework to Accelerate Neural Video Coding](https://mlsanity.com/paper/2606.04595/) (2026)
- [MambaRaw: Selective State Space Modeling for Efficient 4K Raw Image Reconstruction](https://mlsanity.com/paper/2606.24479/) (2026)

## Abstract (arXiv)

Unsupervised video semantic compression (UVSC), i.e., compressing videos to better support various analysis tasks, has recently garnered attention. However, the semantic richness of previous methods remains limited, due to the single semantic learning objective, limited training data, etc. To address this, we propose to boost the UVSC task by absorbing the off-the-shelf rich semantics from VFMs. Specifically, we introduce a VFMs-shared semantic alignment layer, complemented by VFM-specific prompts, to flexibly align semantics between the compressed video and various VFMs. This allows different VFMs to collaboratively build a mutually-enhanced semantic space, guiding the learning of the compression model. Moreover, we introduce a dynamic trajectory-based inter-frame compression scheme, which first estimates the semantic trajectory based on the historical content, and then traverses along the trajectory to predict the future semantics as the coding context. This reduces the overall bitcost of the system, further improving the compression efficiency. Our approach outperforms previous coding methods on three mainstream tasks and six datasets.