# Mixture-of-Experts-based Entropy Model for Learned Image Compression

arXiv: https://arxiv.org/abs/2608.10947 · HTML: https://mlsanity.com/paper/2608.10947/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in MoE large language models, one of 12 directions in Mixture-of-experts.
- Builds on 30 of its 40 references that are in the map.
- First posted on arXiv 11 Aug 2026.

## Placed in

- Mixture-of-experts / [MoE large language models](https://mlsanity.com/moe/moe-large-language-models/)

## Builds on (most cited first, 30 of 30)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [RWKV: Reinventing RNNs for the Transformer Era](https://mlsanity.com/paper/2305.13048/) (2023)
- [Scaling Vision with Sparse Mixture of Experts](https://mlsanity.com/paper/2106.05974/) (2021)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [ELIC: Efficient Learned Image Compression with Unevenly Grouped Space-Channel Contextual Adaptive Coding](https://mlsanity.com/paper/2203.10886/) (2022)
- [Lossy Image Compression with Compressive Autoencoders](https://mlsanity.com/paper/1703.00395/) (2017)
- [Channel-wise Autoregressive Entropy Models for Learned Image Compression](https://mlsanity.com/paper/2007.08739/) (2020)
- [From Sparse to Soft Mixtures of Experts](https://mlsanity.com/paper/2308.00951/) (2023)
- [Learned Image Compression with Mixed Transformer-CNN Architectures](https://mlsanity.com/paper/2303.14978/) (2023)
- [Checkerboard Context Model for Efficient Learned Image Compression](https://mlsanity.com/paper/2103.15306/) (2021)
- [OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models](https://mlsanity.com/paper/2402.01739/) (2024)
- [Efficient Frequency Domain-based Transformers for High-Quality Image Deblurring](https://mlsanity.com/paper/2211.12250/) (2022)
- [The Devil Is in the Details: Window-based Attention for Image Compression](https://mlsanity.com/paper/2203.08450/) (2022)
- [Entroformer: A Transformer-based Entropy Model for Learned Image Compression](https://mlsanity.com/paper/2202.05492/) (2022)
- [MLIC: Multi-Reference Entropy Model for Learned Image Compression](https://mlsanity.com/paper/2211.07273/) (2022)
- [Improving Inference for Neural Image Compression](https://mlsanity.com/paper/2006.04240/) (2020)
- [Complexity Experts are Task-Discriminative Learners for Any Image Restoration](https://mlsanity.com/paper/2411.18466/) (2024)
- [MLIC++: Linear Complexity Multi-Reference Entropy Modeling for Learned Image Compression](https://mlsanity.com/paper/2307.15421/) (2023)
- [Learned Image Compression with Dictionary-based Entropy Model](https://mlsanity.com/paper/2504.00496/) (2025)
- [MambaVC: Learned Visual Compression with Selective State Spaces](https://mlsanity.com/paper/2405.15413/) (2024)
- [Linear Attention Modeling for Learned Image Compression](https://mlsanity.com/paper/2502.05741/) (2025)
- [Learned Image Compression with Hierarchical Progressive Context Modeling](https://mlsanity.com/paper/2507.19125/) (2025)
- [WM-MoE: Weather-aware Multi-scale Mixture-of-Experts for Blind Adverse Weather Removal](https://mlsanity.com/paper/2303.13739/) (2023)
- [MLICv2: Enhanced Multi-Reference Entropy Modeling for Learned Image Compression](https://mlsanity.com/paper/2504.19119/) (2025)

## Abstract (arXiv)

Learned image compression has seen significant progress in recent years with the development of end-to-end learned models that achieve better compression efficiency than state-of-the-art conventional methods. Recently, Mixture of Experts (MoE) approaches have seen promising results in NLP and computer vision tasks. In this paper, we introduce the MoE approach to learned image compression. We propose a MoE-based Entropy model (MoEE) for learned image compression, allowing the model to selectively activate only the subset of parameters required for the input image. Our model achieves a BD-Rate improvement over VVC of -16.85% on the Kodak dataset.