# When Are Tree Structures Necessary for Deep Learning of Representations?

arXiv: https://arxiv.org/abs/1503.00185 · HTML: https://mlsanity.com/paper/1503.00185/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Feb 2015.

## Built on by (most cited first, 6 of 6)

- [Interpreting Deep Learning Models in Natural Language Processing: A Review](https://mlsanity.com/paper/2110.10470/) (2021)
- [Dialogue History Matters! Personalized Response Selectionin Multi-turn Retrieval-based Chatbots](https://mlsanity.com/paper/2103.09534/) (2021)
- [Implicit N-grams Induced by Recurrence](https://mlsanity.com/paper/2205.02724/) (2022)
- [AutoEncoding Tree for City Generation and Applications](https://mlsanity.com/paper/2309.15941/) (2023)
- [A More Efficient Chinese Named Entity Recognition base on BERT and Syntactic Analysis](https://mlsanity.com/paper/2101.11423/) (2021)
- [Accurate Portraits of Scientific Resources and Knowledge Service Components](https://mlsanity.com/paper/2204.04883/) (2022)

## Abstract (arXiv)

Recursive neural models, which use syntactic parse trees to recursively generate representations bottom-up, are a popular architecture. But there have not been rigorous evaluations showing for exactly which tasks this syntax-based method is appropriate. In this paper we benchmark bf recursive neural models against sequential bf recurrent neural models (simple recurrent and LSTM models), enforcing apples-to-apples comparison as much as possible. We investigate 4 tasks: (1) sentiment classification at the sentence level and phrase level; (2) matching questions to answer-phrases; (3) discourse parsing; (4) semantic relation extraction (e.g., em component-whole between nouns). Our goal is to understand better when, and why, recursive models can outperform simpler models. We find that recursive models help mainly on tasks (like semantic relation extraction) that require associating headwords across a long distance, particularly on very long sequences. We then introduce a method for allowing recurrent models to achieve similar performance: breaking long sentences into clause-like units at punctuation and processing them separately before combining. Our results thus help understand the limitations of both classes of models, and suggest directions for improving recurrent models.