# Transition-Based Dependency Parsing with Stack Long Short-Term Memory

arXiv: https://arxiv.org/abs/1505.08075 · HTML: https://mlsanity.com/paper/1505.08075/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 May 2015.

## Built on by (most cited first, 30 of 45)

- [Structured Prediction as Translation between Augmented Natural Languages](https://mlsanity.com/paper/2101.05779/) (2021)
- [Paradigm Shift in Natural Language Processing](https://mlsanity.com/paper/2109.12575/) (2021)
- [Pushing the Limits of ChatGPT on NLP Tasks](https://mlsanity.com/paper/2306.09719/) (2023)
- [Mastering the Explicit Opinion-role Interaction: Syntax-aided Neural Transition System for Unified Opinion Role Labeling](https://mlsanity.com/paper/2110.02001/) (2021)
- [End-to-end Semantic Role Labeling with Neural Transition-based Model](https://mlsanity.com/paper/2101.00394/) (2021)
- [Dependency Parsing as MRC-based Span-Span Prediction](https://mlsanity.com/paper/2105.07654/) (2021)
- [Capitalization and Punctuation Restoration: a Survey](https://mlsanity.com/paper/2111.10746/) (2021)
- [Sequential Cross-Document Coreference Resolution](https://mlsanity.com/paper/2104.08413/) (2021)
- [Effective Batching for Recurrent Neural Network Grammars](https://mlsanity.com/paper/2105.14822/) (2021)
- [SpEL: Structured Prediction for Entity Linking](https://mlsanity.com/paper/2310.14684/) (2023)
- [Generative Pretrained Structured Transformers: Unsupervised Syntactic Language Models at Scale](https://mlsanity.com/paper/2403.08293/) (2024)
- [Recent Trends in Named Entity Recognition (NER)](https://mlsanity.com/paper/2101.11420/) (2021)
- [A Little Pretraining Goes a Long Way: A Case Study on Dependency Parsing Task for Low-resource Morphologically Rich Languages](https://mlsanity.com/paper/2102.06551/) (2021)
- [Few-Shot Semantic Parsing for New Predicates](https://mlsanity.com/paper/2101.10708/) (2021)
- [Transcribing Natural Languages for The Deaf via Neural Editing Programs](https://mlsanity.com/paper/2112.09600/) (2021)
- [Improving Zero-Shot Entity Retrieval through Effective Dense Representations](https://mlsanity.com/paper/2103.04156/) (2021)
- [ANEC: An Amharic Named Entity Corpus and Transformer Based Recognizer](https://mlsanity.com/paper/2207.00785/) (2022)
- [Diversity from Human Feedback](https://mlsanity.com/paper/2310.06648/) (2023)
- [Transformers as Graph-to-Graph Models](https://mlsanity.com/paper/2310.17936/) (2023)
- [Medical Dialogue: A Survey of Categories, Methods, Evaluation and Challenges](https://mlsanity.com/paper/2405.10630/) (2024)
- [Systematic Investigation of Strategies Tailored for Low-Resource Settings for Low-Resource Dependency Parsing](https://mlsanity.com/paper/2201.11374/) (2022)
- [Depression Detection Using Digital Traces on Social Media: A Knowledge-aware Deep Learning Approach](https://mlsanity.com/paper/2303.05389/) (2023)
- [Syntactic Nuclei in Dependency Parsing -- A Multilingual Exploration](https://mlsanity.com/paper/2101.11959/) (2021)
- [Scene Graph Modification as Incremental Structure Expanding](https://mlsanity.com/paper/2209.09093/) (2022)
- [Sentence-Incremental Neural Coreference Resolution](https://mlsanity.com/paper/2305.16947/) (2023)
- [The Emergence of Chunking Structures with Hierarchical RNN](https://mlsanity.com/paper/2309.04919/) (2023)
- [Step-by-step Instructions and a Simple Tabular Output Format Improve the Dependency Parsing Accuracy of LLMs](https://mlsanity.com/paper/2506.09983/) (2025)
- [In-Order Chart-Based Constituent Parsing](https://mlsanity.com/paper/2102.04065/) (2021)
- [The DCU-EPFL Enhanced Dependency Parser at the IWPT 2021 Shared Task](https://mlsanity.com/paper/2107.01982/) (2021)
- [Dependency Parsing with Backtracking using Deep Reinforcement Learning](https://mlsanity.com/paper/2206.13914/) (2022)

## Abstract (arXiv)

We propose a technique for learning representations of parser states in transition-based dependency parsers. Our primary innovation is a new control structure for sequence-to-sequence neural networks---the stack LSTM. Like the conventional stack data structures used in transition-based parsing, elements can be pushed to or popped from the top of the stack in constant time, but, in addition, an LSTM maintains a continuous space embedding of the stack contents. This lets us formulate an efficient parsing model that captures three facets of a parser's state: (i) unbounded look-ahead into the buffer of incoming words, (ii) the complete history of actions taken by the parser, and (iii) the complete contents of the stack of partially built tree fragments, including their internal structures. Standard backpropagation techniques are used for training and yield state-of-the-art parsing performance.