mlsanityData to 14 Oct 2026

Entroformer: A Transformer-based Entropy Model for Learned Image Compression

arXiv 2202.05492 ↗First posted 11 Feb 2022eess.IV cs.CVcited by 107

Not on the map. Why?

Abstract (arXiv)

One critical component in lossy deep image compression is the entropy model, which predicts the probability distribution of the quantized latent representation in the encoding and decoding modules. Previous works build entropy models upon convolutional neural networks which are inefficient in capturing global dependencies. In this work, we propose a novel transformer-based entropy model, termed Entroformer, to capture long-range dependencies in probability distribution estimation effectively and efficiently. Different from vision transformers in image classification, the Entroformer is highly optimized for image compression, including a top-k self-attention and a diamond relative position encoding. Meanwhile, we further expand this architecture with a parallel bidirectional context model to speed up the decoding process. The experiments show that the Entroformer achieves state-of-the-art performance on image compression while being time-efficient.

Key facts. Cited by 107 papers in the map (data through 14 Oct 2026). Builds on 22 of its 32 references that are in the map. First posted on arXiv 11 Feb 2022.

Builds on

22 of 32 references.

References · 22

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2018Self-Attention with Relative Position Representationscited by 1,120
  5. 2018Variational image compression with a scale hyperpriorcited by 935
  6. 2016End-to-end Optimized Image Compressioncited by 696
  7. 2018Joint Autoregressive and Hierarchical Priors for Learned Image Compressioncited by 664
  8. 2020Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modulescited by 590
  9. 2017Lossy Image Compression with Compressive Autoencoderscited by 294
  10. 2020Channel-wise Autoregressive Entropy Models for Learned Image Compressioncited by 264
  11. 2021Checkerboard Context Model for Efficient Learned Image Compressioncited by 202
  12. 2018Context-adaptive Entropy Model for End-to-end Optimized Image Compressioncited by 143
10 more
  1. 2018Conditional Probability Models for Deep Image Compressioncited by 133
  2. 2019Non-local Attention Optimized Deep Image Compressioncited by 92
  3. 2017Learning Convolutional Networks for Content-weighted Image Compressioncited by 76
  4. 2016End-to-end optimization of nonlinear transform codes for perceptual qualitycited by 73
  5. 2020Causal Contextual Prediction for Learned Image Compressioncited by 73
  6. 2021KVT: k-NN Attention for Boosting Vision Transformerscited by 65
  7. 2020Learning Accurate Entropy Model with Global Reference for Image Compressioncited by 45
  8. 2022Ada-NETS: Face Clustering via Adaptive Neighbour Discovery in the Structure Spacecited by 16
  9. 2021Interpolation variable rate image compressioncited by 12
  10. 2020Learning Context-Based Non-local Entropy Modeling for Image Compressioncited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 10; 2023: 25; 2024: 34; 2025: 22; 2026: 16.

107 papers.

Sort:

By area:

  1. 2023Learned Image Compression with Mixed Transformer-CNN Architecturescited by 211
  2. 2023Neural Video Compression with Diverse Contextscited by 165
  3. 2024Neural Video Compression with Feature Modulationcited by 123
  4. 2022MLIC: Multi-Reference Entropy Model for Learned Image Compressioncited by 84
  5. 2022VCT: A Video Compression Transformercited by 74
  6. 2024Towards Extreme Image Compression with Latent Feature Guidance and Diffusion Priorcited by 60
  7. 2023MLIC++: Linear Complexity Multi-Reference Entropy Modeling for Learned Image Compressioncited by 58
  8. 2023Frequency-Aware Transformer for Learned Image Compressioncited by 58
  9. 2023EVC: Towards Real-Time Neural Image Compression with Mask Decaycited by 49
  10. 2022Contextformer: A Transformer with Spatio-Channel Attention for Context Modeling in Learned Image Compressioncited by 45
  11. 2024Lossy Image Compression with Foundation Diffusion Modelscited by 36
  12. 2022Image Compression with Product Quantized Masked Image Modelingcited by 33
38 more
  1. 2025StableCodec: Taming One-Step Diffusion for Extreme Image Compressioncited by 33
  2. 2024HEMGS: A Hybrid Entropy Model for 3D Gaussian Splatting Data Compressioncited by 31
  3. 2025Learned Image Compression with Dictionary-based Entropy Modelcited by 30
  4. 2024MambaVC: Learned Visual Compression with Selective State Spacescited by 28
  5. 2024Image Compression for Machine and Human Vision with Spatial-Frequency Adaptationcited by 26
  6. 2024RDEIC: Accelerating Diffusion-Based Extreme Image Compression with Relay Residual Diffusioncited by 23
  7. 2025Linear Attention Modeling for Learned Image Compressioncited by 23
  8. 2022Deep Lossy Plus Residual Coding for Lossless and Near-lossless Image Compressioncited by 22
  9. 2024CRA5: Extreme Compression of ERA5 for Portable Global Climate and Weather Research via an Efficient Variational Transformercited by 22
  10. 2023Context-Based Trit-Plane Coding for Progressive Image Compressioncited by 21
  11. 2025Learned Image Compression with Hierarchical Progressive Context Modelingcited by 21
  12. 2022High-Efficiency Lossy Image Coding Through Adaptive Neighborhood Information Aggregationcited by 20
  13. 2025MambaIC: State Space Models for High-Performance Learned Image Compressioncited by 20
  14. 2024Causal Context Adjustment Loss for Learned Image Compressioncited by 19
  15. 2024Controlling Rate, Distortion, and Realism: Towards a Single Comprehensive Neural Image Compression Modelcited by 17
  16. 2024Once-for-All: Controllable Generative Image Compression with Dynamic Granularity Adaptationcited by 16
  17. 2022A Comprehensive Survey of Transformers for Computer Visioncited by 14
  18. 2023M2T: Masking Transformers Twice for Faster Decodingcited by 14
  19. 2023Frequency Disentangled Features in Neural Image Compressioncited by 14
  20. 2023Efficient Contextformer: Spatio-Channel Window Attention for Fast Context Modeling in Learned Image Compressioncited by 13
  21. 2023End-to-End Learnable Multi-Scale Feature Compression for VCMcited by 13
  22. 2023Fast and High-Performance Learned Image Compression With Improved Checkerboard Context Model, Deformable Residual Module, and Knowledge Distillationcited by 12
  23. 2022Rate-Distortion Optimized Post-Training Quantization for Learned Image Compressioncited by 10
  24. 2023Dynamic Low-Rank Instance Adaptation for Universal Neural Image Compressioncited by 10
  25. 2025Lossy Neural Compression for Geospatial Analytics: A Reviewcited by 10
  26. 2022High-Fidelity Variable-Rate Image Compression via Invertible Activation Transformationcited by 9
  27. 2023ECSIC: Epipolar Cross Attention for Stereo Image Compressioncited by 8
  28. 2024Generalized Gaussian Model for Learned Image Compressioncited by 8
  29. 2025Bridging the Gap between Gaussian Diffusion Models and Universal Quantization for Image Compressioncited by 8
  30. 2025NeuCo-Bench: A Novel Benchmark Framework for Neural Embeddings in Earth Observationcited by 7
  31. 2022Disparity-based Stereo Image Compression with Aligned Cross-View Priorscited by 6
  32. 2023LLIC: Large Receptive Field Transform Coding with Adaptive Weights for Learned Image Compressioncited by 6
  33. 2024Another Way to the Top: Exploit Contextual Clustering in Learned Image Codingcited by 6
  34. 2024GroupedMixer: An Entropy Model with Group-wise Token-Mixers for Learned Image Compressioncited by 6
  35. 2025Content-Aware Mamba for Learned Image Compressioncited by 6
  36. 2023Multistage Spatial Context Models for Learned Image Compressioncited by 5
  37. 2023Neural Image Compression Using Masked Sparse Visual Representationcited by 5
  38. 2024DeepHQ: Learned Hierarchical Quantizer for Progressive Deep Image Codingcited by 5

50 most cited of 107 All 107 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2021Joint Global and Local Hierarchical Priors for Learned Image Compression
  2. 2024An Information-Theoretic Regularizer for Lossy Neural Image Compression
  3. 2026Efficient Learned Image Compression without Entropy Coding
  4. 2025Efficient Learned Image Compression Through Knowledge Distillation
  5. 2023Image Compression using only Attention based Neural Networks
  6. 2021Learned transform compression with optimized entropy encoding
  7. 2026Efficient training for compact compression models via sequential distillation
  8. 2023AICT: An Adaptive Image Compression Transformer