mlsanityData to 14 Oct 2026

Distillation of a tractable model from the VQ-VAE

arXiv 2509.01400 ↗First posted 1 Sep 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Deep generative models with discrete latent space, such as the Vector-Quantized Variational Autoencoder (VQ-VAE), offer excellent data generation capabilities, but, due to the large size of their latent space, their probabilistic inference is deemed intractable. We demonstrate that the VQ-VAE can be distilled into a tractable model by selecting a subset of latent variables with high probabilities. This simple strategy is particularly efficient, especially if the VQ-VAE underutilizes its latent space, which is, indeed, very often the case. We frame the distilled model as a probabilistic circuit, and show that it preserves expressiveness of the VQ-VAE while providing tractable probabilistic inference. Experiments illustrate competitive performance in density estimation and conditional generation tasks, challenging the view of the VQ-VAE as an inherently intractable model.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 23 of its 37 references that are in the map. First posted on arXiv 1 Sep 2025.

Builds on

23 of 37 references.

References · 23

  1. 2017Neural Discrete Representation Learningcited by 4,712
  2. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
  3. 2016Conditional Image Generation with PixelCNN Decoderscited by 862
  4. 2016Pixel Recurrent Neural Networkscited by 816
  5. 2021SoundStream: An End-to-End Neural Audio Codeccited by 771
  6. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 658
  7. 2020Jukebox: A Generative Model for Musiccited by 542
  8. 2015A note on the evaluation of generative modelscited by 292
  9. 2021Generating Diverse Structure for Image Inpainting With Hierarchical VQ-VAEcited by 106
  10. 2020Einsum Networks: Fast and Scalable Learning of Tractable Probabilistic Circuitscited by 103
  11. 2023Straightening Out the Straight-Through Estimator: Overcoming Optimization Challenges in Vector Quantized Networkscited by 82
  12. 2018Theory and Experiments on Vector Quantized Autoencoderscited by 55
11 more
  1. 2020Robust Training of Vector Quantized Bottleneck Modelscited by 54
  2. 20244M-21: An Any-to-Any Vision Model for Tens of Tasks and Modalitiescited by 52
  3. 2017Generating High-Quality and Informative Conversation Responses with Sequence-to-Sequence Modelscited by 42
  4. 2016On the Latent Variable Interpretation in Sum-Product Networkscited by 40
  5. 2022Scaling Up Probabilistic Circuits by Latent Variable Distillationcited by 35
  6. 2016A Unified Approach for Learning the Parameters of Sum-Product Networkscited by 24
  7. 2020Efficient Marginalization of Discrete and Structured Latent Variables via Sparsitycited by 21
  8. 2022Continuous Mixtures of Tractable Probabilistic Modelscited by 19
  9. 2022Semantic Image Synthesis with Semantically Coupled VQ-Modelcited by 4
  10. 2021Analytically Tractable Inference in Deep Neural Networkscited by 3
  11. 2024Addressing Index Collapse of Large-Codebook Speech Tokenizer with Dual-Decoding Product-Quantized Variational Auto-Encodercited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026VP-VAE: Rethinking Vector Quantization via Adaptive Vector Perturbation
  2. 2025From Classical Probabilistic Latent Variable Models to Modern Generative AI: A Unified Perspective
  3. 2023Understanding the Distillation Process from Deep Generative Models to Tractable Probabilistic Circuits
  4. 2022Robust Vector Quantized-Variational Autoencoder
  5. 2022Diffusion bridges vector quantized Variational AutoEncoders
  6. 2024LASERS: LAtent Space Encoding for Representations with Sparsity for Generative Modeling
  7. 2025Geometry-Preserving Encoder/Decoder in Latent Generative Models
  8. 2025A Frequentist Statistical Introduction to Variational Inference, Autoencoders, and Diffusion Models