# Discrete Latent Variable Representations for Low-Resource Text Classification

arXiv: https://arxiv.org/abs/2006.06226 · HTML: https://mlsanity.com/paper/2006.06226/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jun 2020.

## Built on by (most cited first, 10 of 10)

- [Latent Diffusion Energy-Based Model for Interpretable Text Modeling](https://mlsanity.com/paper/2206.05895/) (2022)
- [Latent Space Energy-Based Model of Symbol-Vector Coupling for Text Generation and Classification](https://mlsanity.com/paper/2108.11556/) (2021)
- [StreamHover: Livestream Transcript Summarization and Annotation](https://mlsanity.com/paper/2109.05160/) (2021)
- [I Predict Therefore I Am: Is Next Token Prediction Enough to Learn Human-Interpretable Concepts from Data?](https://mlsanity.com/paper/2503.08980/) (2025)
- [Prototypical Fine-tuning: Towards Robust Performance Under Varying Data Sizes](https://mlsanity.com/paper/2211.13638/) (2022)
- [Autoregressive Co-Training for Learning Discrete Speech Representations](https://mlsanity.com/paper/2203.15840/) (2022)
- [Large Language Model as Token Compressor and Decompressor](https://mlsanity.com/paper/2603.25340/) (2026)
- [Entropy optimized semi-supervised decomposed vector-quantized variational autoencoder model based on transfer learning for multiclass text classification and generation](https://mlsanity.com/paper/2111.08453/) (2021)
- [Learning Semantic Textual Similarity via Topic-informed Discrete Latent Variables](https://mlsanity.com/paper/2211.03616/) (2022)
- [A Vector-Quantized Foundation Model for Patient Behavior Monitoring](https://mlsanity.com/paper/2503.15221/) (2025)

## Abstract (arXiv)

While much work on deep latent variable models of text uses continuous latent variables, discrete latent variables are interesting because they are more interpretable and typically more space efficient. We consider several approaches to learning discrete latent variable models for text in the case where exact marginalization over these variables is intractable. We compare the performance of the learned representations as features for low-resource document and sentence classification. Our best models outperform the previous best reported results with continuous representations in these low-resource settings, while learning significantly more compressed representations. Interestingly, we find that an amortized variant of Hard EM performs particularly well in the lowest-resource regimes.