# Transition-based Parsing with Stack-Transformers

arXiv: https://arxiv.org/abs/2010.10669 · HTML: https://mlsanity.com/paper/2010.10669/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Oct 2020.

## Built on by (most cited first, 30 of 38)

- [Graph Pre-training for AMR Parsing and Generation](https://mlsanity.com/paper/2203.07836/) (2022)
- [Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scale](https://mlsanity.com/paper/2203.00633/) (2022)
- [Maximum Bayes Smatch Ensemble Distillation for AMR Parsing](https://mlsanity.com/paper/2112.07790/) (2021)
- [AMR Parsing with Action-Pointer Transformer](https://mlsanity.com/paper/2104.14674/) (2021)
- [Structural Guidance for Transformer Language Models](https://mlsanity.com/paper/2108.00104/) (2021)
- [Structure-aware Fine-tuning of Sequence-to-sequence Transformers for Transition-based AMR Parsing](https://mlsanity.com/paper/2110.15534/) (2021)
- [A Two-Stream AMR-enhanced Model for Document-level Event Argument Extraction](https://mlsanity.com/paper/2205.00241/) (2022)
- [Cost-effective End-to-end Information Extraction for Semi-structured Document Images](https://mlsanity.com/paper/2104.08041/) (2021)
- [Inducing and Using Alignments for Transition-based AMR Parsing](https://mlsanity.com/paper/2205.01464/) (2022)
- [Enhanced Knowledge Selection for Grounded Dialogues via Document Semantic Graphs](https://mlsanity.com/paper/2206.07296/) (2022)
- [Hierarchical Curriculum Learning for AMR Parsing](https://mlsanity.com/paper/2110.07855/) (2021)
- [Enhancing Document-level Event Argument Extraction with Contextual Clues and Role Relevance](https://mlsanity.com/paper/2310.05991/) (2023)
- [Probabilistic, Structure-Aware Algorithms for Improved Variety, Accuracy, and Coverage of AMR Alignments](https://mlsanity.com/paper/2106.06002/) (2021)
- [Bootstrapping Multilingual AMR with Contextual Word Alignments](https://mlsanity.com/paper/2102.02189/) (2021)
- [An AMR-based Link Prediction Approach for Document-level Event Argument Extraction](https://mlsanity.com/paper/2305.19162/) (2023)
- [Dependency Transformer Grammars: Integrating Dependency Structures into Transformer Language Models](https://mlsanity.com/paper/2407.17406/) (2024)
- [Incorporating Graph Information in Transformer-based AMR Parsing](https://mlsanity.com/paper/2306.13467/) (2023)
- [Text2MDT: Extracting Medical Decision Trees from Medical Texts](https://mlsanity.com/paper/2401.02034/) (2024)
- [ATP: AMRize Then Parse! Enhancing AMR Parsing with PseudoAMRs](https://mlsanity.com/paper/2204.08875/) (2022)
- [Discontinuous Grammar as a Foreign Language](https://mlsanity.com/paper/2110.10431/) (2021)
- [Learning to Transpile AMR into SPARQL](https://mlsanity.com/paper/2112.07877/) (2021)
- [Improve Event Extraction via Self-Training with Gradient Guidance](https://mlsanity.com/paper/2205.12490/) (2022)
- [Approximating CKY with Transformers](https://mlsanity.com/paper/2305.02386/) (2023)
- [Levi Graph AMR Parser using Heterogeneous Attention](https://mlsanity.com/paper/2107.04152/) (2021)
- [Cross-domain Generalization for AMR Parsing](https://mlsanity.com/paper/2210.12445/) (2022)
- [On the Challenges of Fully Incremental Neural Dependency Parsing](https://mlsanity.com/paper/2309.16254/) (2023)
- [CIC-BART-SSA: Controllable Image Captioning with Structured Semantic Augmentation](https://mlsanity.com/paper/2407.11393/) (2024)
- [Emergent Stack Representations in Modeling Counter Languages Using Transformers](https://mlsanity.com/paper/2502.01432/) (2025)
- [SoilNet: A Multimodal Multitask Model for Hierarchical Classification of Soil Horizons](https://mlsanity.com/paper/2508.03785/) (2025)
- [Structured Sentiment Analysis as Transition-based Dependency Graph Parsing](https://mlsanity.com/paper/2305.05311/) (2023)

## Abstract (arXiv)

Modeling the parser state is key to good performance in transition-based parsing. Recurrent Neural Networks considerably improved the performance of transition-based systems by modelling the global state, e.g. stack-LSTM parsers, or local state modeling of contextualized features, e.g. Bi-LSTM parsers. Given the success of Transformer architectures in recent parsing systems, this work explores modifications of the sequence-to-sequence Transformer architecture to model either global or local parser states in transition-based parsing. We show that modifications of the cross attention mechanism of the Transformer considerably strengthen performance both on dependency and Abstract Meaning Representation (AMR) parsing tasks, particularly for smaller models or limited training data.