# Representations of Syntax [MASK] Useful: Effects of Constituency and Dependency Structure in Recursive LSTMs

arXiv: https://arxiv.org/abs/2005.00019 · HTML: https://mlsanity.com/paper/2005.00019/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Apr 2020.

## Built on by (most cited first, 4 of 4)

- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [LasUIE: Unifying Information Extraction with Latent Adaptive Structure-aware Generative Language Model](https://mlsanity.com/paper/2304.06248/) (2023)
- [How poor is the stimulus? Evaluating hierarchical generalization in neural networks trained on child-directed speech](https://mlsanity.com/paper/2301.11462/) (2023)
- [Punctuation Restoration Improves Structure Understanding Without Supervision](https://mlsanity.com/paper/2402.08382/) (2024)

## Abstract (arXiv)

Sequence-based neural networks show significant sensitivity to syntactic structure, but they still perform less well on syntactic tasks than tree-based networks. Such tree-based networks can be provided with a constituency parse, a dependency parse, or both. We evaluate which of these two representational schemes more effectively introduces biases for syntactic structure that increase performance on the subject-verb agreement prediction task. We find that a constituency-based network generalizes more robustly than a dependency-based one, and that combining the two types of structure does not yield further improvement. Finally, we show that the syntactic robustness of sequential models can be substantially improved by fine-tuning on a small amount of constructed data, suggesting that data augmentation is a viable alternative to explicit constituency structure for imparting the syntactic biases that sequential models are lacking.