MixCompress: Mixture of Experts for Variable Rate Learned Image Compression

arXiv 2607.14334 ↗First posted 15 Jul 2026cs.CVcited by 0

On the map: Mixture-of-experts → MoE large language models

Abstract (arXiv)

Learned image compression (LIC) is bottlenecked by the need to store independent models for each rate-distortion operating point. Existing variable bit-rate (VBR) methods aim to reduce this overhead via dense parameter modulation, but forcing a shared backbone to approximate divergent mappings causes severe feature entanglement. Specifically, low-rate smoothing gradients inherently conflict with the preservation of high-frequency textural details, leading to sub-optimal performance. To resolve this, we propose MixCompress, a unified VBR framework based on sparse structural specialization. While sparsely gated Mixture-of-Experts (MoE) routing successfully mitigates gradient conflict, it operates on a fixed computational budget. To address the increased representational demands of higher bit-rates we introduce a Mixture-of-Depths (MoD) extension to dynamically scale model capacity. Combined with Conditional Auxiliary Transforms (CAT) for dynamic sub-band energy modulation, our hierarchical framework effectively dynamically scales capacity. Extensive evaluations demonstrate that MixCompress not only matches individually optimized single-rate baselines but can even surpass them, establishing a new Pareto frontier for computationally efficient image coding.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MoE large language models, one of 12 directions in Mixture-of-experts. Builds on 24 of its 30 references that are in the map. First posted on arXiv 15 Jul 2026.

Builds on

24 of 30 references.

Closest ancestors · 2

Same direction, newest first

Other references · 22

2016Layer Normalizationcited by 5,579
2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
2020Gradient Surgery for Multi-Task Learningcited by 1,210
2018Variational image compression with a scale hyperpriorcited by 910
2018Multi-Task Learning as Multi-Objective Optimizationcited by 809
2021Scaling Vision with Sparse Mixture of Expertscited by 740
2016End-to-end Optimized Image Compressioncited by 673
2018Joint Autoregressive and Hierarchical Priors for Learned Image Compressioncited by 642
2020Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modulescited by 570
2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 441
2022ELIC: Efficient Learned Image Compression with Unevenly Grouped Space-Channel Contextual Adaptive Codingcited by 296
2023Learned Image Compression with Mixed Transformer-CNN Architecturescited by 205

10 more

2021Checkerboard Context Model for Efficient Learned Image Compressioncited by 193
2022On the Representation Collapse of Sparse Mixture of Expertscited by 156
2019Variable Rate Deep Image Compression With a Conditional Autoencodercited by 109
2020Asymmetric Gained Deep Image Compression With Continuous Rate Adaptationcited by 83
2023Frequency-Aware Transformer for Learned Image Compressioncited by 56
2019Variable Rate Deep Image Compression with Modulated Autoencodercited by 39
2025Linear Attention Modeling for Learned Image Compressioncited by 22
2023QVRF: A Quantization-error-aware Variable Rate Framework for Learned Image Compressioncited by 11
2022Edge-Aware Autoencoder Design for Real-Time Mixture-of-Experts Image Compressioncited by 8
2024DeepFGS: Fine-Grained Scalable Coding for Learned Image Compressioncited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way