# Improving Robustness and Generality of NLP Models Using Disentangled Representations

arXiv: https://arxiv.org/abs/2009.09587 · HTML: https://mlsanity.com/paper/2009.09587/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Sep 2020.

## Built on by (most cited first, 8 of 8)

- [Disentangled Representation Learning](https://mlsanity.com/paper/2211.11695/) (2022)
- [Closed-Loop Unsupervised Representation Disentanglement with β-VAE Distillation and Diffusion Probabilistic Feedback](https://mlsanity.com/paper/2402.02346/) (2024)
- [Causal Disentanglement for Regulating Social Influence Bias in Social Recommendation](https://mlsanity.com/paper/2403.03578/) (2024)
- [Causal Prototype-inspired Contrast Adaptation for Unsupervised Domain Adaptive Semantic Segmentation of High-resolution Remote Sensing Imagery](https://mlsanity.com/paper/2403.03704/) (2024)
- [Enhancing Visual In-Context Learning by Multi-Faceted Fusion](https://mlsanity.com/paper/2601.10107/) (2026)
- [Towards Robust and Reliable Concept Representations: Reliability-Enhanced Concept Embedding Model](https://mlsanity.com/paper/2502.01191/) (2025)
- [Disentangling Score Content and Performance Style for Joint Piano Rendering and Transcription](https://mlsanity.com/paper/2509.23878/) (2025)
- [Decafs: Disentangled Conditional adversarial Flows](https://mlsanity.com/paper/2607.18755/) (2026)

## Abstract (arXiv)

Supervised neural networks, which first map an input x to a single representation z, and then map z to the output label y, have achieved remarkable success in a wide range of natural language processing (NLP) tasks. Despite their success, neural models lack for both robustness and generality: small perturbations to inputs can result in absolutely different outputs; the performance of a model trained on one domain drops drastically when tested on another domain. In this paper, we present methods to improve robustness and generality of NLP models from the standpoint of disentangled representation learning. Instead of mapping x to a single representation z, the proposed strategy maps x to a set of representations z1,z2,...,zK while forcing them to be disentangled. These representations are then mapped to different logits ls, the ensemble of which is used to make the final prediction y. We propose different methods to incorporate this idea into currently widely-used models, including adding an L2 regularizer on zs or adding Total Correlation (TC) under the framework of variational information bottleneck (VIB). We show that models trained with the proposed criteria provide better robustness and domain adaptation ability in a wide range of supervised learning tasks.