mlsanityData to 7 Oct 2026

M2T: Masking Transformers Twice for Faster Decoding

arXiv 2304.07313 ↗First posted 14 Apr 2023eess.IV cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

We show how bidirectional transformers trained for masked token prediction can be applied to neural image compression to achieve state-of-the-art results. Such models were previously used for image generation by progressivly sampling groups of masked tokens according to uncertainty-adaptive schedules. Unlike these works, we demonstrate that predefined, deterministic schedules perform as well or better for image compression. This insight allows us to use masked attention during training in addition to masked inputs, and activation caching during inference, to significantly speed up our models (~4 higher inference speed) at a small increase in bitrate.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 31 of its 41 references that are in the map. First posted on arXiv 14 Apr 2023.

Builds on

31 of 41 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  5. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  6. 2018Variational image compression with a scale hyperpriorcited by 910
  7. 2020On Layer Normalization in the Transformer Architecturecited by 906
  8. 2016End-to-end Optimized Image Compressioncited by 673
  9. 2018Joint Autoregressive and Hierarchical Priors for Learned Image Compressioncited by 642
  10. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  11. 2020Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modulescited by 570
  12. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
19 more
  1. 2017PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modificationscited by 415
  2. 2020High-Fidelity Generative Image Compressioncited by 331
  3. 2022ELIC: Efficient Learned Image Compression with Unevenly Grouped Space-Channel Contextual Adaptive Codingcited by 296
  4. 2017Lossy Image Compression with Compressive Autoencoderscited by 289
  5. 2020Channel-wise Autoregressive Entropy Models for Learned Image Compressioncited by 255
  6. 2018Generative Adversarial Networks for Extreme Learned Image Compressioncited by 213
  7. 2019Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoffcited by 207
  8. 2021Checkerboard Context Model for Efficient Learned Image Compressioncited by 193
  9. 2022The Devil Is in the Details: Window-based Attention for Image Compressioncited by 141
  10. 2020Nonlinear Transform Codingcited by 132
  11. 2018Conditional Probability Models for Deep Image Compressioncited by 128
  12. 2022Entroformer: A Transformer-based Entropy Model for Learned Image Compressioncited by 102
  13. 2022Multi-Realism Image Compression with a Conditional Generatorcited by 79
  14. 2022An Introduction to Neural Data Compressioncited by 78
  15. 2022VCT: A Video Compression Transformercited by 74
  16. 2016End-to-end optimization of nonlinear transform codes for perceptual qualitycited by 72
  17. 2018Deep Generative Models for Distribution-Preserving Lossy Compressioncited by 59
  18. 2022Contextformer: A Transformer with Spatio-Channel Attention for Context Modeling in Learned Image Compressioncited by 42
  19. 2022Image Compression with Product Quantized Masked Image Modelingcited by 32

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 3; 2025: 6; 2026: 1.

14 papers.

Sort:

  1. 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 425
  2. 2023GIVT: Generative Infinite-Vocabulary Transformerscited by 80
  3. 2025Halton Scheduler For Masked Generative Image Transformercited by 30
  4. 2025Learned Image Compression with Dictionary-based Entropy Modelcited by 29
  5. 2025Learned Image Compression with Hierarchical Progressive Context Modelingcited by 20
  6. 2024Causal Context Adjustment Loss for Learned Image Compressioncited by 19
  7. 2024Controlling Rate, Distortion, and Realism: Towards a Single Comprehensive Neural Image Compression Modelcited by 16
  8. 2023MobileNVC: Real-time 1080p Neural Video Compression on a Mobile Devicecited by 11
  9. 2024Generalized Gaussian Model for Learned Image Compressioncited by 8
  10. 2026Differentiable Vector Quantization for Rate-Distortion Optimization of Generative Image Compressioncited by 4
  11. 2025Context Guided Transformer Entropy Modeling for Video Compressioncited by 3
  12. 2023Local Masking Meets Progressive Freezing: Crafting Efficient Vision Transformers for Self-Supervised Learningcited by 1
2 more
  1. 2025PerCoV2: Ultra-Low Bit-Rate Perceptual Image Compression via Query-Based 1D Multimodal Image Tokenscited by 0
  2. 2025TreeNet: A Light Weight Model for Low Bitrate Image Compressioncited by 0