# Dependency Parsing with Bottom-up Hierarchical Pointer Networks

arXiv: https://arxiv.org/abs/2105.09611 · HTML: https://mlsanity.com/paper/2105.09611/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 49 references that are in the map.
- First posted on arXiv 20 May 2021.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF](https://mlsanity.com/paper/1603.01354/) (2016)
- [Deep Biaffine Attention for Neural Dependency Parsing](https://mlsanity.com/paper/1611.01734/) (2016)
- [Syntax-BERT: Improving Pre-trained Transformers with Syntax Trees](https://mlsanity.com/paper/2103.04350/) (2021)
- [Efficient Second-Order TreeCRF for Neural Dependency Parsing](https://mlsanity.com/paper/2005.00975/) (2020)
- [Do Syntax Trees Help Pre-trained Transformers Extract Information?](https://mlsanity.com/paper/2008.09084/) (2020)
- [Head-Driven Phrase Structure Grammar Parsing on Penn Treebank](https://mlsanity.com/paper/1907.02684/) (2019)
- [Stack-Pointer Networks for Dependency Parsing](https://mlsanity.com/paper/1805.01087/) (2018)
- [Anaphora and Coreference Resolution: A Review](https://mlsanity.com/paper/1805.11824/) (2018)
- [Left-to-Right Dependency Parsing with Pointer Networks](https://mlsanity.com/paper/1903.08445/) (2019)
- [Recursive Non-Autoregressive Graph-to-Graph Transformer for Dependency Parsing with Iterative Refinement](https://mlsanity.com/paper/2003.13118/) (2020)
- [Polyglot: Distributed Word Representations for Multilingual NLP](https://mlsanity.com/paper/1307.1662/) (2013)
- [Second-Order Neural Dependency Parsing with Message Passing and End-to-End Training](https://mlsanity.com/paper/2010.05003/) (2020)
- [Interpretable Visual Question Answering by Reasoning on Dependency Trees](https://mlsanity.com/paper/1809.01810/) (2018)
- [Dependency Parsing as Head Selection](https://mlsanity.com/paper/1606.01280/) (2016)
- [Deep Contextualized Word Embeddings in Transition-Based and Graph-Based Dependency Parsing -- A Tale of Two Parsers Revisited](https://mlsanity.com/paper/1908.07397/) (2019)
- [Multitask Pointer Network for Multi-Representational Parsing](https://mlsanity.com/paper/2009.09730/) (2020)
- [Syntax-Enhanced Neural Machine Translation with Syntax-Aware Word Representations](https://mlsanity.com/paper/1905.02878/) (2019)
- [Hierarchical Pointer Net Parsing](https://mlsanity.com/paper/1908.11571/) (2019)
- [Global Greedy Dependency Parsing](https://mlsanity.com/paper/1911.08673/) (2019)
- [Neural Probabilistic Model for Non-projective MST Parsing](https://mlsanity.com/paper/1701.00874/) (2017)
- [From POS tagging to dependency parsing for biomedical event extraction](https://mlsanity.com/paper/1808.03731/) (2018)

## Built on by (most cited first, 5 of 5)

- [Bottom-Up Constituency Parsing and Nested Named Entity Recognition with Pointer Networks](https://mlsanity.com/paper/2110.05419/) (2021)
- [Headed-Span-Based Projective Dependency Parsing](https://mlsanity.com/paper/2108.04750/) (2021)
- [Combining (second-order) graph-based and headed-span-based projective dependency parsing](https://mlsanity.com/paper/2108.05838/) (2021)
- [On the Challenges of Fully Incremental Neural Dependency Parsing](https://mlsanity.com/paper/2309.16254/) (2023)
- [An evaluation of LLMs for generating movie reviews: GPT-4o, Gemini-2.0 and DeepSeek-V3](https://mlsanity.com/paper/2506.00312/) (2025)

## Abstract (arXiv)

Dependency parsing is a crucial step towards deep language understanding and, therefore, widely demanded by numerous Natural Language Processing applications. In particular, left-to-right and top-down transition-based algorithms that rely on Pointer Networks are among the most accurate approaches for performing dependency parsing. Additionally, it has been observed for the top-down algorithm that Pointer Networks' sequential decoding can be improved by implementing a hierarchical variant, more adequate to model dependency structures. Considering all this, we develop a bottom-up-oriented Hierarchical Pointer Network for the left-to-right parser and propose two novel transition-based alternatives: an approach that parses a sentence in right-to-left order and a variant that does it from the outside in. We empirically test the proposed neural architecture with the different algorithms on a wide variety of languages, outperforming the original approach in practically all of them and setting new state-of-the-art results on the English and Chinese Penn Treebanks for non-contextualized and BERT-based embeddings.