mlsanityData to 7 Oct 2026

Disentangling Generative Factors in Natural Language with Discrete Variational Autoencoders

arXiv 2109.07169 ↗First posted 15 Sep 2021cs.CLcited by 19

Not on the map. Why?

Abstract (arXiv)

The ability of learning disentangled representations represents a major step for interpretable NLP systems as it allows latent linguistic features to be controlled. Most approaches to disentanglement rely on continuous variables, both for images and text. We argue that despite being suitable for image datasets, continuous variables may not be ideal to model features of textual data, due to the fact that most generative factors in text are discrete. We propose a Variational Autoencoder based method which models language features as discrete variables and encourages independence between variables for learning disentangled representations. The proposed model outperforms continuous and discrete baselines on several qualitative and quantitative benchmarks for disentanglement as well as on a text style transfer downstream application.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 21 of its 35 references that are in the map. First posted on arXiv 15 Sep 2021.

Builds on

21 of 35 references.

References · 21

  1. 2017Neural Discrete Representation Learningcited by 4,650
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  4. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  5. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 858
  6. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 719
  7. 2018Disentangling by Factorisingcited by 614
  8. 2015Generating Sentences from a Continuous Spacecited by 607
  9. 2018Understanding disentangling in β-VAEcited by 332
  10. 2017Toward Controlled Generation of Textcited by 271
  11. 2017Style Transfer from Non-Parallel Text by Cross-Alignmentcited by 246
  12. 2018Isolating Sources of Disentanglement in Variational Autoencoderscited by 167
9 more
  1. 2019PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variablecited by 145
  2. 2018Disentangled Representation Learning for Non-Parallel Text Style Transfercited by 127
  3. 2018Learning Disentangled Joint Continuous and Discrete Representationscited by 83
  4. 2020Improving Disentangled Text Representation Learning with Information-Theoretic Guidancecited by 52
  5. 2021A Novel Estimator of Mutual Information for Learning to Disentangle Textual Representationscited by 37
  6. 2019Implicit Deep Latent Variable Models for Text Generationcited by 26
  7. 2020Evidence-Aware Inferential Text Generation with Vector Quantised Variational AutoEncodercited by 6
  8. 2020Controllable Text Generation with Focused Variationcited by 6
  9. 2020Multi-type Disentanglement without Adversarial Trainingcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 3; 2024: 6; 2025: 4; 2026: 1.

19 papers.

Sort:

By area:

  1. 2022Learning Disentangled Representations for Natural Language Definitionscited by 12
  2. 2024Graph-based Unsupervised Disentangled Representation Learning via Multimodal Large Language Modelscited by 9
  3. 2022Vector-Quantized Input-Contextualized Soft Prompts for Natural Language Understandingcited by 7
  4. 2023Learning Disentangled Semantic Spaces of Explanations via Invertible Neural Networkscited by 6
  5. 2024Improving Semantic Control in Discrete Latent Spaces with Transformer Quantized Variational Autoencoderscited by 6
  6. 2024Are there identifiable structural parts in the sentence embedding whole?cited by 6
  7. 2022Quasi-symbolic Semantic Geometry over Transformer-based Variational AutoEncodercited by 5
  8. 2022Blackbird's language matrices (BLMs): a new benchmark to investigate disentangled generalisation in neural networkscited by 4
  9. 2025LangVAE and LangSpace: Building and Probing for Language Model VAEscited by 3
  10. 2025Characterizing Mamba's Selective Memory using Auto-Encoderscited by 3
  11. 2024Interpreting Latent Student Knowledge Representations in Programming Assignmentscited by 2
  12. 2025Testing the assumptions about the geometry of sentence embedding spaces: the cosine measure need not applycited by 2
7 more
  1. 2022An Overview on Controllable Text Generation via Variational Auto-Encoderscited by 1
  2. 2023Language as a Latent Sequence: deep latent variable models for semi-supervised paraphrase generationcited by 1
  3. 2024Disentanglement with Factor Quantized Variational Autoencoderscited by 1
  4. 2024ACCEPT: Adaptive Codebook for Composite and Efficient Prompt Tuningcited by 1
  5. 2023Disentangling continuous and discrete linguistic signals in transformer-based sentence embeddingscited by 0
  6. 2025Bridging Compositional and Distributional Semantics: A Survey on Latent Semantic Geometry via AutoEncodercited by 0
  7. 2026Uncovering Latent Reasoning Strategies in Language Modelscited by 0