# Syntax-Infused Variational Autoencoder for Text Generation

arXiv: https://arxiv.org/abs/1906.02181 · HTML: https://mlsanity.com/paper/1906.02181/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jun 2019.

## Built on by (most cited first, 19 of 19)

- [Generating Syntactically Controlled Paraphrases without Using Annotated Parallel Pairs](https://mlsanity.com/paper/2101.10579/) (2021)
- [The Homogenizing Effect of Large Language Models on Human Expression and Thought](https://mlsanity.com/paper/2508.01491/) (2025)
- [Semantic Diversity in Dialogue with Natural Language Inference](https://mlsanity.com/paper/2205.01497/) (2022)
- [MOVER: Mask, Over-generate and Rank for Hyperbole Generation](https://mlsanity.com/paper/2109.07726/) (2021)
- [ParaAMR: A Large-Scale Syntactically Diverse Paraphrase Dataset by AMR Back-Translation](https://mlsanity.com/paper/2305.16585/) (2023)
- [Unsupervised Contextual Paraphrase Generation using Lexical Control and Reinforcement Learning](https://mlsanity.com/paper/2103.12777/) (2021)
- [HyperSDFusion: Bridging Hierarchical Structures in Language and Geometry for Enhanced 3D Text2Shape Generation](https://mlsanity.com/paper/2403.00372/) (2024)
- [Learning Disentangled Semantic Representations for Zero-Shot Cross-Lingual Transfer in Multilingual Machine Reading Comprehension](https://mlsanity.com/paper/2204.00996/) (2022)
- [Graph-Induced Syntactic-Semantic Spaces in Transformer-Based Variational AutoEncoders](https://mlsanity.com/paper/2311.08579/) (2023)
- [Towards Unsupervised Content Disentanglement in Sentence Representations via Syntactic Roles](https://mlsanity.com/paper/2206.11184/) (2022)
- [Unsupervised Syntactically Controlled Paraphrase Generation with Abstract Meaning Representations](https://mlsanity.com/paper/2211.00881/) (2022)
- [Exploiting Inductive Bias in Transformers for Unsupervised Disentanglement of Syntax and Semantics with VAEs](https://mlsanity.com/paper/2205.05943/) (2022)
- [Cooking Up Creativity: Enhancing LLM Creativity through Structured Recombination](https://mlsanity.com/paper/2504.20643/) (2025)
- [Dependency-based Mixture Language Models](https://mlsanity.com/paper/2203.10256/) (2022)
- [Transformers as Neural Augmentors: Class Conditional Sentence Generation via Variational Bayes](https://mlsanity.com/paper/2205.09391/) (2022)
- [Coherence and Diversity through Noise: Self-Supervised Paraphrase Generation via Structure-Aware Denoising](https://mlsanity.com/paper/2302.02780/) (2023)
- [Pragmatically Appropriate Diversity for Dialogue Evaluation](https://mlsanity.com/paper/2304.02812/) (2023)
- [Interpretable Sentence Representation with Variational Autoencoders and Attention](https://mlsanity.com/paper/2305.02810/) (2023)
- [Bridging Compositional and Distributional Semantics: A Survey on Latent Semantic Geometry via AutoEncoder](https://mlsanity.com/paper/2506.20083/) (2025)

## Abstract (arXiv)

We present a syntax-infused variational autoencoder (SIVAE), that integrates sentences with their syntactic trees to improve the grammar of generated sentences. Distinct from existing VAE-based text generative models, SIVAE contains two separate latent spaces, for sentences and syntactic trees. The evidence lower bound objective is redesigned correspondingly, by optimizing a joint distribution that accommodates two encoders and two decoders. SIVAE works with long short-term memory architectures to simultaneously generate sentences and syntactic trees. Two versions of SIVAE are proposed: one captures the dependencies between the latent variables through a conditional prior network, and the other treats the latent variables independently such that syntactically-controlled sentence generation can be performed. Experimental results demonstrate the generative superiority of SIVAE on both reconstruction and targeted syntactic evaluations. Finally, we show that the proposed models can be used for unsupervised paraphrasing given different syntactic tree templates.