# Learning Disentangled Semantic Spaces of Explanations via Invertible Neural Networks

arXiv: https://arxiv.org/abs/2305.01713 · HTML: https://mlsanity.com/paper/2305.01713/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 87 references that are in the map.
- First posted on arXiv 2 May 2023.

## Builds on (most cited first, 30 of 47)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [What you can cram into a single vector: Probing sentence embeddings for linguistic properties](https://mlsanity.com/paper/1805.01070/) (2018)
- [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://mlsanity.com/paper/1705.02364/) (2017)
- [MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance](https://mlsanity.com/paper/1909.02622/) (2019)
- [Adversarial Autoencoders](https://mlsanity.com/paper/1511.05644/) (2015)
- [AllenNLP: A Deep Semantic Natural Language Processing Platform](https://mlsanity.com/paper/1803.07640/) (2018)
- [Compositionality decomposed: how do neural networks generalise?](https://mlsanity.com/paper/1908.08351/) (2019)
- [DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations](https://mlsanity.com/paper/2006.03659/) (2020)
- [ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer](https://mlsanity.com/paper/2105.11741/) (2021)
- [Isolating Sources of Disentanglement in Variational Autoencoders](https://mlsanity.com/paper/1802.04942/) (2018)
- [Explaining Answers with Entailment Trees](https://mlsanity.com/paper/2104.08661/) (2021)
- [A Tutorial on Principal Component Analysis](https://mlsanity.com/paper/1404.1100/) (2014)
- [DiffCSE: Difference-based Contrastive Learning for Sentence Embeddings](https://mlsanity.com/paper/2204.10298/) (2022)
- [Disentangled Representation Learning for Non-Parallel Text Style Transfer](https://mlsanity.com/paper/1808.04339/) (2018)
- [TSDAE: Using Transformer-based Sequential Denoising Auto-Encoder for Unsupervised Sentence Embedding Learning](https://mlsanity.com/paper/2104.06979/) (2021)
- [Optimus: Organizing Sentences via Pre-trained Modeling of a Latent Space](https://mlsanity.com/paper/2004.04092/) (2020)
- [Learning Deep Disentangled Embeddings with the F-Statistic Loss](https://mlsanity.com/paper/1802.05312/) (2018)
- [Deep Learning of Representations: Looking Forward](https://mlsanity.com/paper/1305.0445/) (2013)
- [Measuring Disentanglement: A Review of Metrics](https://mlsanity.com/paper/2012.09276/) (2020)
- [WorldTree: A Corpus of Explanation Graphs for Elementary Science Questions supporting Multi-Hop Inference](https://mlsanity.com/paper/1802.03052/) (2018)
- [Generating Sentences from Disentangled Syntactic and Semantic Spaces](https://mlsanity.com/paper/1907.05789/) (2019)
- [The paradox of the compositionality of natural language: a neural machine translation case study](https://mlsanity.com/paper/2108.05885/) (2021)
- [A Disentangling Invertible Interpretation Network for Explaining Latent Representations](https://mlsanity.com/paper/2004.13166/) (2020)
- [Learning Disentangled Representations in the Imaging Domain](https://mlsanity.com/paper/2108.12043/) (2021)

## Built on by (most cited first, 6 of 6)

- [Nexus: An Omni-Perceptive And -Interactive Model for Language, Audio, And Vision](https://mlsanity.com/paper/2503.01879/) (2025)
- [Graph-Induced Syntactic-Semantic Spaces in Transformer-Based Variational AutoEncoders](https://mlsanity.com/paper/2311.08579/) (2023)
- [Improving Semantic Control in Discrete Latent Spaces with Transformer Quantized Variational Autoencoders](https://mlsanity.com/paper/2402.00723/) (2024)
- [LlaMaVAE: Guiding Large Language Model Generation via Continuous Latent Sentence Spaces](https://mlsanity.com/paper/2312.13208/) (2023)
- [Disentangled Concepts Speak Louder Than Words: Explainable Video Action Recognition](https://mlsanity.com/paper/2511.03725/) (2025)
- [Towards Controllable Natural Language Inference through Lexical Inference Types](https://mlsanity.com/paper/2308.03581/) (2023)

## Abstract (arXiv)

Disentangled latent spaces usually have better semantic separability and geometrical properties, which leads to better interpretability and more controllable data generation. While this has been well investigated in Computer Vision, in tasks such as image disentanglement, in the NLP domain sentence disentanglement is still comparatively under-investigated. Most previous work have concentrated on disentangling task-specific generative factors, such as sentiment, within the context of style transfer. In this work, we focus on a more general form of sentence disentanglement, targeting the localised modification and control of more general sentence semantic features. To achieve this, we contribute to a novel notion of sentence semantic disentanglement and introduce a flow-based invertible neural network (INN) mechanism integrated with a transformer-based language Autoencoder (AE) in order to deliver latent spaces with better separability properties. Experimental results demonstrate that the model can conform the distributed latent space into a better semantically disentangled sentence space, leading to improved language interpretability and controlled generation when compared to the recent state-of-the-art language VAE models.