# Learning to Compose Task-Specific Tree Structures

arXiv: https://arxiv.org/abs/1707.02786 · HTML: https://mlsanity.com/paper/1707.02786/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jul 2017.

## Built on by (most cited first, 30 of 48)

- [A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learning](https://mlsanity.com/paper/2110.01515/) (2021)
- [LasUIE: Unifying Information Extraction with Latent Adaptive Structure-aware Generative Language Model](https://mlsanity.com/paper/2304.06248/) (2023)
- [Improving Cross-Modal Alignment in Vision Language Navigation via Syntactic Information](https://mlsanity.com/paper/2104.09580/) (2021)
- [Learning Algebraic Recombination for Compositional Generalization](https://mlsanity.com/paper/2107.06516/) (2021)
- [Bridging Discrete and Backpropagation: Straight-Through and Beyond](https://mlsanity.com/paper/2304.08612/) (2023)
- [DABERT: Dual Attention Enhanced BERT for Semantic Matching](https://mlsanity.com/paper/2210.03454/) (2022)
- [Improving Semantic Matching through Dependency-Enhanced Pre-trained Model with Adaptive Fusion](https://mlsanity.com/paper/2210.08471/) (2022)
- [Modeling Hierarchical Structures with Continuous Recursive Neural Networks](https://mlsanity.com/paper/2106.06038/) (2021)
- [Fast-R2D2: A Pretrained Recursive Neural Network based on Pruned CKY for Grammar Induction and Text Representation](https://mlsanity.com/paper/2203.00281/) (2022)
- [Conversational Semantic Role Labeling with Predicate-Oriented Latent Graph](https://mlsanity.com/paper/2210.03037/) (2022)
- [ChordMixer: A Scalable Neural Attention Model for Sequences with Different Lengths](https://mlsanity.com/paper/2206.05852/) (2022)
- [Comateformer: Combined Attention Transformer for Semantic Sentence Matching](https://mlsanity.com/paper/2412.07220/) (2024)
- [Training Discrete Deep Generative Models via Gapped Straight-Through Estimator](https://mlsanity.com/paper/2206.07235/) (2022)
- [CHEF: Cross-modal Hierarchical Embeddings for Food Domain Retrieval](https://mlsanity.com/paper/2102.02547/) (2021)
- [Beam Tree Recursive Cells](https://mlsanity.com/paper/2305.19999/) (2023)
- [Augmenting Transformers with Recursively Composed Multi-grained Representations](https://mlsanity.com/paper/2309.16319/) (2023)
- [Recursion in Recursion: Two-Level Nested Recursion for Length Generalization with Scalability](https://mlsanity.com/paper/2311.04449/) (2023)
- [Learning Syntax from Naturally-Occurring Bracketings](https://mlsanity.com/paper/2104.13933/) (2021)
- [Rule Augmented Unsupervised Constituency Parsing](https://mlsanity.com/paper/2105.10193/) (2021)
- [A Multi-Grained Self-Interpretable Symbolic-Neural Model For Single/Multi-Labeled Text Classification](https://mlsanity.com/paper/2303.02860/) (2023)
- [Structured Dialogue Discourse Parsing](https://mlsanity.com/paper/2306.15103/) (2023)
- [Efficient Beam Tree Recursion](https://mlsanity.com/paper/2307.10779/) (2023)
- [Deep Learning-based Sentiment Classification: A Comparative Survey](https://mlsanity.com/paper/2312.17253/) (2023)
- [Forming Trees with Treeformers](https://mlsanity.com/paper/2207.06960/) (2022)
- [Weakly Supervised Reasoning by Neuro-Symbolic Approaches](https://mlsanity.com/paper/2309.13072/) (2023)
- [Increasing The Performance of Cognitively Inspired Data-Efficient Language Models via Implicit Structure Building](https://mlsanity.com/paper/2310.20589/) (2023)
- [Tree Transformers are an Ineffective Model of Syntactic Constituency](https://mlsanity.com/paper/2411.16993/) (2024)
- [Learning with Latent Structures in Natural Language Processing: A Survey](https://mlsanity.com/paper/2201.00490/) (2022)
- [StrAE: Autoencoding for Pre-Trained Embeddings using Explicit Structure](https://mlsanity.com/paper/2305.05588/) (2023)
- [The Emergence of Chunking Structures with Hierarchical RNN](https://mlsanity.com/paper/2309.04919/) (2023)

## Abstract (arXiv)

For years, recursive neural networks (RvNNs) have been shown to be suitable for representing text into fixed-length vectors and achieved good performance on several natural language processing tasks. However, the main drawback of RvNNs is that they require structured input, which makes data preparation and model implementation hard. In this paper, we propose Gumbel Tree-LSTM, a novel tree-structured long short-term memory architecture that learns how to compose task-specific tree structures only from plain text data efficiently. Our model uses Straight-Through Gumbel-Softmax estimator to decide the parent node among candidates dynamically and to calculate gradients of the discrete decision. We evaluate the proposed model on natural language inference and sentiment analysis, and show that our model outperforms or is at least comparable to previous models. We also find that our model converges significantly faster than other models.