# Exploration into Translation-Equivariant Image Quantization

arXiv: https://arxiv.org/abs/2112.00384 · HTML: https://mlsanity.com/paper/2112.00384/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 22 references that are in the map.
- First posted on arXiv 1 Dec 2021.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [CogView: Mastering Text-to-Image Generation via Transformers](https://mlsanity.com/paper/2105.13290/) (2021)
- [Vector-quantized Image Modeling with Improved VQGAN](https://mlsanity.com/paper/2110.04627/) (2021)
- [Making Convolutional Networks Shift-Invariant Again](https://mlsanity.com/paper/1904.11486/) (2019)
- [Why do deep convolutional networks generalize so poorly to small image transformations?](https://mlsanity.com/paper/1805.12177/) (2018)
- [Delving Deeper into Anti-aliasing in ConvNets](https://mlsanity.com/paper/2008.09604/) (2020)
- [Impact of Aliasing on Generalization in Deep Convolutional Networks](https://mlsanity.com/paper/2108.03489/) (2021)

## Built on by (most cited first, 11 of 11)

- [GFT: Graph Foundation Model with Transferable Tree Vocabulary](https://mlsanity.com/paper/2411.06070/) (2024)
- [ADDP: Learning General Representations for Image Recognition and Generation with Alternating Denoising Diffusion Process](https://mlsanity.com/paper/2306.05423/) (2023)
- [Representation Collapsing Problems in Vector Quantization](https://mlsanity.com/paper/2411.16550/) (2024)
- [Identifying Spatio-Temporal Drivers of Extreme Events](https://mlsanity.com/paper/2410.24075/) (2024)
- [VP-VAE: Rethinking Vector Quantization via Adaptive Vector Perturbation](https://mlsanity.com/paper/2602.17133/) (2026)
- [Beyond Stationarity: Rethinking Codebook Collapse in Vector Quantization](https://mlsanity.com/paper/2602.18896/) (2026)
- [Early Quantization Shrinks Codebook: A Simple Fix for Diversity-Preserving Tokenization](https://mlsanity.com/paper/2603.17052/) (2026)
- [Identifying and Clustering Counter Relationships of Team Compositions in PvP Games for Efficient Balance Analysis](https://mlsanity.com/paper/2408.17180/) (2024)
- [VQEL: Enabling Self-Play in Emergent Language Games via Agent-Internal Vector Quantization](https://mlsanity.com/paper/2503.04940/) (2025)
- [Hierarchical Spatial Algorithms for High-Resolution Image Quantization and Feature Extraction](https://mlsanity.com/paper/2510.08449/) (2025)
- [ProTok: Unified Protein Tokenization via Modular Multi-Codebook Quantization](https://mlsanity.com/paper/2610.18481/) (2026)

## Abstract (arXiv)

This is an exploratory study that discovers the current image quantization (vector quantization) do not satisfy translation equivariance in the quantized space due to aliasing. Instead of focusing on anti-aliasing, we propose a simple yet effective way to achieve translation-equivariant image quantization by enforcing orthogonality among the codebook embeddings. To explore the advantages of translation-equivariant image quantization, we conduct three proof-of-concept experiments with a carefully controlled dataset: (1) text-to-image generation, where the quantized image indices are the target to predict, (2) image-to-text generation, where the quantized image indices are given as a condition, (3) using a smaller training set to analyze sample efficiency. From the strictly controlled experiments, we empirically verify that the translation-equivariant image quantizer improves not only sample efficiency but also the accuracy over VQGAN up to +11.9% in text-to-image generation and +3.9% in image-to-text generation.