# Homology-constrained vector quantization entropy regularizer

arXiv: https://arxiv.org/abs/2211.14363 · HTML: https://mlsanity.com/paper/2211.14363/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 13 references that are in the map.
- First posted on arXiv 25 Nov 2022.

## Builds on (most cited first, 7 of 7)

- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [Jukebox: A Generative Model for Music](https://mlsanity.com/paper/2005.00341/) (2020)
- [Topological Autoencoders](https://mlsanity.com/paper/1906.00722/) (2019)
- [Can neural networks learn persistent homology features?](https://mlsanity.com/paper/2011.14688/) (2020)
- [Persistent homology for low-complexity models](https://mlsanity.com/paper/1709.01037/) (2017)

## Built on by (most cited first, 3 of 3)

- [MotionDreamer: One-to-Many Motion Synthesis with Localized Generative Masked Transformer](https://mlsanity.com/paper/2504.08959/) (2025)
- [Discrete Tokenization for Multimodal LLMs: A Comprehensive Survey](https://mlsanity.com/paper/2507.22920/) (2025)
- [BioSerenity-E1: a self-supervised EEG model for medical applications](https://mlsanity.com/paper/2503.10362/) (2025)

## Abstract (arXiv)

This paper describes an entropy regularization term for vector quantization (VQ) based on the analysis of persistent homology of the VQ embeddings. Higher embedding entropy positively correlates with higher codebook utilization, mitigating overfit towards the identity and codebook collapse in VQ-based autoencoders [1]. We show that homology-constrained regularization is an effective way to increase entropy of the VQ process (approximated to input entropy) while preserving the approximated topology in the quantized latent space, averaged over mini batches. This work further explores some patterns of persistent homology diagrams of latents formed by vector quantization. We implement and test the proposed algorithm as a module integrated into a sample VQ-VAE. Linked code repository provides a functioning implementation of the proposed architecture, referred to as homology-constrained vector quantization (HC-VQ) further in this work.