# Recursive Non-Autoregressive Graph-to-Graph Transformer for Dependency Parsing with Iterative Refinement

arXiv: https://arxiv.org/abs/2003.13118 · HTML: https://mlsanity.com/paper/2003.13118/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Mar 2020.

## Built on by (most cited first, 20 of 20)

- [A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond](https://mlsanity.com/paper/2204.09269/) (2022)
- [RQUGE: Reference-Free Metric for Evaluating Question Generation by Answering the Question](https://mlsanity.com/paper/2211.01482/) (2022)
- [Dependency Parsing as MRC-based Span-Span Prediction](https://mlsanity.com/paper/2105.07654/) (2021)
- [BenchCLAMP: A Benchmark for Evaluating Language Models on Syntactic and Semantic Parsing](https://mlsanity.com/paper/2206.10668/) (2022)
- [Headed-Span-Based Projective Dependency Parsing](https://mlsanity.com/paper/2108.04750/) (2021)
- [Syntax-Aware Graph-to-Graph Transformer for Semantic Role Labelling](https://mlsanity.com/paper/2104.07704/) (2021)
- [Explanation Graph Generation via Pre-trained Language Models: An Empirical Study with Contrastive Learning](https://mlsanity.com/paper/2204.04813/) (2022)
- [Graph Refinement for Coreference Resolution](https://mlsanity.com/paper/2203.16574/) (2022)
- [Dependency Parsing with Bottom-up Hierarchical Pointer Networks](https://mlsanity.com/paper/2105.09611/) (2021)
- [Combining (second-order) graph-based and headed-span-based projective dependency parsing](https://mlsanity.com/paper/2108.05838/) (2021)
- [Transformers as Graph-to-Graph Models](https://mlsanity.com/paper/2310.17936/) (2023)
- [Multilingual Extraction and Categorization of Lexical Collocations with Graph-aware Transformers](https://mlsanity.com/paper/2205.11456/) (2022)
- [Toward a Graph Foundation Model: Pre-Training Transformers With Random Walks](https://mlsanity.com/paper/2506.14098/) (2025)
- [Order-Level Attention Similarity Across Language Models: A Latent Commonality](https://mlsanity.com/paper/2511.05064/) (2025)
- [GADePo: Graph-Assisted Declarative Pooling Transformers for Document-Level Relation Extraction](https://mlsanity.com/paper/2308.14423/) (2023)
- [High-order Joint Constituency and Dependency Parsing](https://mlsanity.com/paper/2309.11888/) (2023)
- [Thai Universal Dependency Treebank](https://mlsanity.com/paper/2405.07586/) (2024)
- [Fast-and-Frugal Text-Graph Transformers are Effective Link Predictors](https://mlsanity.com/paper/2408.06778/) (2024)
- [CSSL: Contrastive Self-Supervised Learning for Dependency Parsing on Relatively Free Word Ordered and Morphologically Rich Low Resource Languages](https://mlsanity.com/paper/2410.06944/) (2024)
- [Scaling Graph-Based Dependency Parsing with Arc Vectorization and Attention-Based Refinement](https://mlsanity.com/paper/2501.09451/) (2025)

## Abstract (arXiv)

We propose the Recursive Non-autoregressive Graph-to-Graph Transformer architecture (RNGTr) for the iterative refinement of arbitrary graphs through the recursive application of a non-autoregressive Graph-to-Graph Transformer and apply it to syntactic dependency parsing. We demonstrate the power and effectiveness of RNGTr on several dependency corpora, using a refinement model pre-trained with BERT. We also introduce Syntactic Transformer (SynTr), a non-recursive parser similar to our refinement model. RNGTr can improve the accuracy of a variety of initial parsers on 13 languages from the Universal Dependencies Treebanks, English and Chinese Penn Treebanks, and the German CoNLL2009 corpus, even improving over the new state-of-the-art results achieved by SynTr, significantly improving the state-of-the-art for all corpora tested.