# Sandwiched Compression: Repurposing Standard Codecs with Neural Network Wrappers

arXiv: https://arxiv.org/abs/2402.05887 · HTML: https://mlsanity.com/paper/2402.05887/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Feb 2024.

## Built on by (most cited first, 7 of 7)

- [SCALED : Surrogate-gradient for Codec-Aware Learning of Downsampling in ABR Streaming](https://mlsanity.com/paper/2602.00198/) (2026)
- [Versatile Recompression-Aware Perceptual Image Super-Resolution](https://mlsanity.com/paper/2511.18090/) (2025)
- [SCENE: Semantic-aware Codec Enhancement with Neural Embeddings](https://mlsanity.com/paper/2601.22189/) (2026)
- [SEAOTTER: Sensor Embedded Autoencoding with One-Time Transcode for Efficient Reconstruction](https://mlsanity.com/paper/2606.03940/) (2026)
- [A Projection-Based Surrogate Gradient Interpretation for Neural Codec Wrappers](https://mlsanity.com/paper/2606.20671/) (2026)
- [Differentiable Proxy Learning for Adaptive Quantization Control in H.264 Video Coding](https://mlsanity.com/paper/2607.10478/) (2026)
- [Efficient Gaussian Splatting Sequence Compression with Standard Video Codecs](https://mlsanity.com/paper/2610.07795/) (2026)

## Abstract (arXiv)

We propose sandwiching standard image and video codecs between pre- and post-processing neural networks. The networks are jointly trained through a differentiable codec proxy to minimize a given rate-distortion loss. This sandwich architecture not only improves the standard codec's performance on its intended content, but more importantly, adapts the codec to other types of image/video content and to other distortion measures. The sandwich learns to transmit ``neural code images'' that optimize and improve overall rate-distortion performance, with the improvements becoming significant especially when the overall problem is well outside of the scope of the codec's design. We apply the sandwich architecture to standard codecs with mismatched sources transporting different numbers of channels, higher resolution, higher dynamic range, computer graphics, and with perceptual distortion measures. The results demonstrate substantial improvements (up to 9 dB gains or up to 30% bitrate reductions) compared to alternative adaptations. We establish optimality properties for sandwiched compression and design differentiable codec proxies approximating current standard codecs. We further analyze model complexity, visual quality under perceptual metrics, as well as sandwich configurations that offer interesting potentials in video compression and streaming.