mlsanityData to 7 Oct 2026

A Multi-Task Approach for Disentangling Syntax and Semantics in Sentence Representations

arXiv 1904.01173 ↗First posted 2 Apr 2019cs.CLcited by 37

Not on the map. Why?

Abstract (arXiv)

We propose a generative model for a sentence that uses two latent variables, with one intended to represent the syntax of the sentence and the other to represent its semantics. We show we can achieve better disentanglement between semantic and syntactic representations by training with multiple losses, including losses that exploit aligned paraphrastic sentences and word-order information. We also investigate the effect of moving from bag-of-words to recurrent neural network modules. We evaluate our models as well as several popular pretrained embeddings on standard semantic similarity tasks and novel syntactic similarity tasks. Empirically, we find that the model with the best performing syntactic and semantic representations also gives rise to the most disentangled representations.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 11; 2023: 8; 2024: 3; 2025: 5; 2026: 3.

37 papers.

Sort:

By area:

  1. 2023Evaluating Large Language Models on Controlled Generation Taskscited by 73
  2. 2021Generating Syntactically Controlled Paraphrases without Using Annotated Parallel Pairscited by 45
  3. 2023Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systemscited by 37
  4. 2021Disentangling Semantics and Syntax in Sentence Embeddings with Pre-trained Language Modelscited by 22
  5. 2021Factorising Meaning and Form for Intent-Preserving Paraphrasingcited by 22
  6. 2022Hierarchical Sketch Induction for Paraphrase Generationcited by 21
  7. 2022Text Revision by On-the-Fly Representation Optimizationcited by 17
  8. 2021Self-conditioning pre-trained language modelscited by 16
  9. 2021Disentangling Representations of Text by Masking Transformerscited by 15
  10. 2022Learning Disentangled Representations for Natural Language Definitionscited by 12
  11. 2022Debiasing Learning for Membership Inference Attacks Against Recommender Systemscited by 9
  12. 2022Towards Robust and Semantically Organised Latent Representations for Unsupervised Text Style Transfercited by 7
25 more
  1. 2022Learning Disentangled Semantic Representations for Zero-Shot Cross-Lingual Transfer in Multilingual Machine Reading Comprehensioncited by 6
  2. 2023Graph-Induced Syntactic-Semantic Spaces in Transformer-Based Variational AutoEncoderscited by 6
  3. 2024Are there identifiable structural parts in the sentence embedding whole?cited by 6
  4. 2025Decoupling Content and Expression: Two-Dimensional Detection of AI-Generated Textcited by 6
  5. 2022Towards Unsupervised Content Disentanglement in Sentence Representations via Syntactic Rolescited by 5
  6. 2023PIP: Parse-Instructed Prefix for Syntactically Controlled Paraphrase Generationcited by 4
  7. 2024Enhanced Coherence-Aware Network with Hierarchical Disentanglement for Aspect-Category Sentiment Analysiscited by 4
  8. 2021Exploiting Twitter as Source of Large Corpora of Weakly Similar Pairs for Semantic Sentence Embeddingscited by 3
  9. 2022Exploiting Inductive Bias in Transformers for Unsupervised Disentanglement of Syntax and Semantics with VAEscited by 3
  10. 2022RSTGen: Imbuing Fine-Grained Interpretable Control into Long-FormText Generatorscited by 3
  11. 2022Mitigating Negative Style Transfer in Hybrid Dialogue Systemcited by 3
  12. 2023Syntactically Robust Training on Partially-Observed Data for Open Information Extractioncited by 3
  13. 2025Disentangling Feature Structure: A Mathematically Provable Two-Stage Training Dynamics in Transformerscited by 3
  14. 2022Bootstrapping meaning through listening: Unsupervised learning of spoken sentence embeddingscited by 2
  15. 2025Testing the assumptions about the geometry of sentence embedding spaces: the cosine measure need not applycited by 2
  16. 2026Time Travel Engine: A Shared Latent Chronological Manifold Enables Historical Navigation in Large Language Modelscited by 2
  17. 2021Unsupervised Representation Disentanglement of Text: An Evaluation on Synthetic Datasetscited by 1
  18. 2023A Quality-based Syntactic Template Retriever for Syntactically-controlled Paraphrase Generationcited by 1
  19. 2025Discovering Semantic Subdimensions through Disentangled Conceptual Representationscited by 1
  20. 2023Interpretable Sentence Representation with Variational Autoencoders and Attentioncited by 0
  21. 2023Disentangling continuous and discrete linguistic signals in transformer-based sentence embeddingscited by 0
  22. 2024Distilling Monolingual and Crosslingual Word-in-Context Representationscited by 0
  23. 2025Bridging Compositional and Distributional Semantics: A Survey on Latent Semantic Geometry via AutoEncodercited by 0
  24. 2026Triospect: A Three-Dimensional Framework for Robust Statistical AI-Generated Text Detection Against Diverse Attackscited by 0
  25. 2026Disentangled Contrastive Learning for Zero-Shot Multilingual Dense Retrievalcited by 0