# Jointly Learning Sentence Embeddings and Syntax with Unsupervised Tree-LSTMs

arXiv: https://arxiv.org/abs/1705.09189 · HTML: https://mlsanity.com/paper/1705.09189/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 May 2017.

## Built on by (most cited first, 24 of 24)

- [R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling](https://mlsanity.com/paper/2107.00967/) (2021)
- [Modeling Hierarchical Structures with Continuous Recursive Neural Networks](https://mlsanity.com/paper/2106.06038/) (2021)
- [Fast-R2D2: A Pretrained Recursive Neural Network based on Pruned CKY for Grammar Induction and Text Representation](https://mlsanity.com/paper/2203.00281/) (2022)
- [Generative Pretrained Structured Transformers: Unsupervised Syntactic Language Models at Scale](https://mlsanity.com/paper/2403.08293/) (2024)
- ["Is depression related to cannabis?": A knowledge-infused model for Entity and Relation Extraction with Limited Supervision](https://mlsanity.com/paper/2102.01222/) (2021)
- [Beam Tree Recursive Cells](https://mlsanity.com/paper/2305.19999/) (2023)
- [To be Closer: Learning to Link up Aspects with Opinions](https://mlsanity.com/paper/2109.08382/) (2021)
- [Augmenting Transformers with Recursively Composed Multi-grained Representations](https://mlsanity.com/paper/2309.16319/) (2023)
- [Recursion in Recursion: Two-Level Nested Recursion for Length Generalization with Scalability](https://mlsanity.com/paper/2311.04449/) (2023)
- [Rule Augmented Unsupervised Constituency Parsing](https://mlsanity.com/paper/2105.10193/) (2021)
- [A Multi-Grained Self-Interpretable Symbolic-Neural Model For Single/Multi-Labeled Text Classification](https://mlsanity.com/paper/2303.02860/) (2023)
- [Efficient Beam Tree Recursion](https://mlsanity.com/paper/2307.10779/) (2023)
- [Should Semantic Vector Composition be Explicit? Can it be Linear?](https://mlsanity.com/paper/2104.06555/) (2021)
- [Learning compositional structures for semantic graph parsing](https://mlsanity.com/paper/2106.04398/) (2021)
- [Forming Trees with Treeformers](https://mlsanity.com/paper/2207.06960/) (2022)
- [Weakly Supervised Reasoning by Neuro-Symbolic Approaches](https://mlsanity.com/paper/2309.13072/) (2023)
- [Learning with Latent Structures in Natural Language Processing: A Survey](https://mlsanity.com/paper/2201.00490/) (2022)
- [StrAE: Autoencoding for Pre-Trained Embeddings using Explicit Structure](https://mlsanity.com/paper/2305.05588/) (2023)
- [Unsupervised Learning of Explainable Parse Trees for Improved Generalisation](https://mlsanity.com/paper/2104.04998/) (2021)
- [Syntactic Inductive Biases for Deep Learning Methods](https://mlsanity.com/paper/2206.04806/) (2022)
- [Unsupervised Inference of Data-Driven Discourse Structures using a Tree Auto-Encoder](https://mlsanity.com/paper/2210.09559/) (2022)
- [Nondeterministic Stacks in Neural Networks](https://mlsanity.com/paper/2304.12955/) (2023)
- [Unsupervised Morphological Tree Tokenizer](https://mlsanity.com/paper/2406.15245/) (2024)
- [Know Yourself and Know Your Neighbour: A Syntactically Informed Self-Supervised Compositional Sentence Representation Learning Framework using a Recursive Hypernetwork](https://mlsanity.com/paper/2610.16547/) (2026)

## Abstract (arXiv)

We introduce a neural network that represents sentences by composing their words according to induced binary parse trees. We use Tree-LSTM as our composition function, applied along a tree structure found by a fully differentiable natural language chart parser. Our model simultaneously optimises both the composition function and the parser, thus eliminating the need for externally-provided parse trees which are normally required for Tree-LSTM. It can therefore be seen as a tree-based RNN that is unsupervised with respect to the parse trees. As it is fully differentiable, our model is easily trained with an off-the-shelf gradient descent method and backpropagation. We demonstrate that it achieves better performance compared to various supervised Tree-LSTM architectures on a textual entailment task and a reverse dictionary task.