Unsupervised Representation Disentanglement of Text: An Evaluation on Synthetic Datasets
Not on the map. Why?
Abstract (arXiv)
To highlight the challenges of achieving representation disentanglement for text domain in an unsupervised setting, in this paper we select a representative set of successfully applied models from the image domain. We evaluate these models on 6 disentanglement metrics, as well as on downstream classification tasks and homotopy. To facilitate the evaluation, we propose two synthetic datasets with known generative factors. Our experiments highlight the existing gap in the text domain and illustrate that certain elements such as representation sparsity (as an inductive bias), or representation coupling with the decoder could impact disentanglement. To the best of our knowledge, our work is the first attempt on the intersection of unsupervised representation disentanglement and text, and provides the experimental framework and datasets for examining future developments in this direction.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 32 of its 45 references that are in the map. First posted on arXiv 7 Jun 2021.
Builds on
32 of 45 references.
References · 32
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
- 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
- 2018Disentangling by Factorisingcited by 614
- 2019Designing and Interpreting Probes with Control Taskscited by 608
- 2015Generating Sentences from a Continuous Spacecited by 607
- 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 452
- 2018Understanding disentangling in β-VAEcited by 332
- 2018Towards a Definition of Disentangled Representationscited by 294
- 2017Variational Inference of Disentangled Latent Concepts from Unlabeled Observationscited by 240
- 2018Targeted Syntactic Evaluation of Language Modelscited by 239
- 2020Information-Theoretic Probing with Minimum Description Lengthcited by 230
20 more
- 2020Information-Theoretic Probing for Linguistic Structurecited by 204
- 2018Disentangled Representation Learning for Non-Parallel Text Style Transfercited by 127
- 2019Lagging Inference Networks and Posterior Collapse in Variational Autoencoderscited by 124
- 2018Learning Deep Disentangled Embeddings with the F-Statistic Losscited by 101
- 2019Preventing Posterior Collapse with delta-VAEscited by 86
- 2013Deep Learning of Representations: Looking Forwardcited by 84
- 2017Improved Variational Autoencoders for Text Modeling using Dilated Convolutionscited by 81
- 2020On the Transfer of Disentangled Representations in Realistic Settingscited by 63
- 2019Generating Sentences from Disentangled Syntactic and Semantic Spacescited by 59
- 2020Improving Disentangled Text Representation Learning with Information-Theoretic Guidancecited by 52
- 2018Avoiding Latent Variable Collapse With Generative Skip Modelscited by 47
- 2020Intrinsic Probing through Dimension Selectioncited by 45
- 2019A Multi-Task Approach for Disentangling Syntax and Semantics in Sentence Representationscited by 37
- 2018Assessing Composition in Sentence Vector Representationscited by 26
- 2021Benchmarks, Algorithms, and Metrics for Hierarchical Disentanglementcited by 12
- 2019Effective Estimation of Deep Generative Language Modelscited by 9
- 2019On the Importance of the Kullback-Leibler Divergence Term in Variational Autoencoders for Text Generationcited by 9
- 2020Polarized-VAE: Proximity Based Disentangled Representation Learning for Text Generationcited by 7
- 2020Do sequence-to-sequence VAEs learn global features of sentences?cited by 6
- 2020Learning Sparse Sentence Encoding without Supervision: An Exploration of Sparsity in Variational Autoencoderscited by 1
Built on by
Citing papers per year: 2022: 1.
1 paper.