# SCENE: Semantic-aware Codec Enhancement with Neural Embeddings

arXiv: https://arxiv.org/abs/2601.22189 · HTML: https://mlsanity.com/paper/2601.22189/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 21 references that are in the map.
- First posted on arXiv 29 Jan 2026.

## Builds on (most cited first, 13 of 13)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features](https://mlsanity.com/paper/2502.14786/) (2025)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [The Road Less Scheduled](https://mlsanity.com/paper/2405.15682/) (2024)
- [Towards Practical Real-Time Neural Video Compression](https://mlsanity.com/paper/2502.20762/) (2025)
- [Neural Compression-Based Feature Learning for Video Restoration](https://mlsanity.com/paper/2203.09208/) (2022)
- [AsConvSR: Fast and Lightweight Super-Resolution Network with Assembled Convolutions](https://mlsanity.com/paper/2305.03387/) (2023)
- [Sandwiched Compression: Repurposing Standard Codecs with Neural Network Wrappers](https://mlsanity.com/paper/2402.05887/) (2024)
- [Learning Source Disentanglement in Neural Audio Codec](https://mlsanity.com/paper/2409.11228/) (2024)
- [VMAF Re-implementation on PyTorch: Some Experimental Results](https://mlsanity.com/paper/2310.15578/) (2023)
- [BONES: Near-Optimal Neural-Enhanced Video Streaming](https://mlsanity.com/paper/2310.09920/) (2023)
- [A Tri-Dynamic Preprocessing Framework for UGC Video Compression](https://mlsanity.com/paper/2512.16101/) (2025)

## Abstract (arXiv)

Compression artifacts from standard video codecs often degrade perceptual quality. We propose a lightweight, semantic-aware pre-processing framework that enhances perceptual fidelity by selectively addressing these distortions. Our method integrates semantic embeddings from a vision-language model into an efficient convolutional architecture, prioritizing the preservation of perceptually significant structures. The model is trained end-to-end with a differentiable codec proxy, enabling it to mitigate artifacts from various standard codecs without modifying the existing video pipeline. During inference, the codec proxy is discarded, and SCENE operates as a standalone pre-processor, enabling real-time performance. Experiments on high-resolution benchmarks show improved performance over baselines in both objective (MS-SSIM) and perceptual (VMAF) metrics, with notable gains in preserving detailed textures within salient regions. Our results show that semantic-guided, codec-aware pre-processing is an effective approach for enhancing compressed video streams.