# On the Challenges of Fully Incremental Neural Dependency Parsing

arXiv: https://arxiv.org/abs/2309.16254 · HTML: https://mlsanity.com/paper/2309.16254/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 37 references that are in the map.
- First posted on arXiv 28 Sep 2023.

## Builds on (most cited first, 17 of 17)

- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Deep Biaffine Attention for Neural Dependency Parsing](https://mlsanity.com/paper/1611.01734/) (2016)
- [Grammar as a Foreign Language](https://mlsanity.com/paper/1412.7449/) (2014)
- [Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations](https://mlsanity.com/paper/1603.04351/) (2016)
- [Head-Driven Phrase Structure Grammar Parsing on Penn Treebank](https://mlsanity.com/paper/1907.02684/) (2019)
- [Transition-based Parsing with Stack-Transformers](https://mlsanity.com/paper/2010.10669/) (2020)
- [Viable Dependency Parsing as Sequence Labeling](https://mlsanity.com/paper/1902.10505/) (2019)
- [A Targeted Assessment of Incremental Processing in Neural LanguageModels and Humans](https://mlsanity.com/paper/2106.03232/) (2021)
- [Constituent Parsing as Sequence Labeling](https://mlsanity.com/paper/1810.08994/) (2018)
- [Probing for Incremental Parse States in Autoregressive Language Models](https://mlsanity.com/paper/2211.09748/) (2022)
- [Strongly Incremental Constituency Parsing with Graph Neural Networks](https://mlsanity.com/paper/2010.14568/) (2020)
- [Bracketing Encodings for 2-Planar Dependency Parsing](https://mlsanity.com/paper/2011.00596/) (2020)
- [Incremental Processing in the Age of Non-Incremental Encoders: An Empirical Assessment of Bidirectional Models for Incremental NLU](https://mlsanity.com/paper/2010.05330/) (2020)
- [A Modest Pareto Optimisation Analysis of Dependency Parsers in 2021](https://mlsanity.com/paper/2106.04216/) (2021)
- [Dependency Parsing with Bottom-up Hierarchical Pointer Networks](https://mlsanity.com/paper/2105.09611/) (2021)
- [An Incremental Turn-Taking Model For Task-Oriented Dialog Systems](https://mlsanity.com/paper/1905.11806/) (2019)
- [A Full Non-Monotonic Transition System for Unrestricted Non-Projective Parsing](https://mlsanity.com/paper/1706.03367/) (2017)

## Built on by (most cited first, 2 of 2)

- [From Partial to Strictly Incremental Constituent Parsing](https://mlsanity.com/paper/2402.02782/) (2024)
- [Bringing Emerging Architectures to Sequence Labeling in NLP](https://mlsanity.com/paper/2509.25918/) (2025)

## Abstract (arXiv)

Since the popularization of BiLSTMs and Transformer-based bidirectional encoders, state-of-the-art syntactic parsers have lacked incrementality, requiring access to the whole sentence and deviating from human language processing. This paper explores whether fully incremental dependency parsing with modern architectures can be competitive. We build parsers combining strictly left-to-right neural encoders with fully incremental sequence-labeling and transition-based decoders. The results show that fully incremental parsing with modern architectures considerably lags behind bidirectional parsing, noting the challenges of psycholinguistically plausible parsing.