# Schema-Free Dependency Parsing via Sequence Generation

arXiv: https://arxiv.org/abs/2201.12407 · HTML: https://mlsanity.com/paper/2201.12407/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 57 references that are in the map.
- First posted on arXiv 28 Jan 2022.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [AllenNLP: A Deep Semantic Natural Language Processing Platform](https://mlsanity.com/paper/1803.07640/) (2018)
- [Deep Biaffine Attention for Neural Dependency Parsing](https://mlsanity.com/paper/1611.01734/) (2016)
- [Injecting Numerical Reasoning Skills into Language Models](https://mlsanity.com/paper/2004.04487/) (2020)
- [Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?](https://mlsanity.com/paper/2005.00628/) (2020)
- [Graph-Structured Representations for Visual Question Answering](https://mlsanity.com/paper/1609.05600/) (2016)
- [Semi-Supervised Sequence Modeling with Cross-View Training](https://mlsanity.com/paper/1809.08370/) (2018)
- [Efficient Second-Order TreeCRF for Neural Dependency Parsing](https://mlsanity.com/paper/2005.00975/) (2020)
- [Simpler but More Accurate Semantic Dependency Parsing](https://mlsanity.com/paper/1807.01396/) (2018)
- [Stack-Pointer Networks for Dependency Parsing](https://mlsanity.com/paper/1805.01087/) (2018)
- [Viable Dependency Parsing as Sequence Labeling](https://mlsanity.com/paper/1902.10505/) (2019)
- [Broad-Coverage Semantic Parsing as Transduction](https://mlsanity.com/paper/1909.02607/) (2019)
- [Enhancing Machine Translation with Dependency-Aware Self-Attention](https://mlsanity.com/paper/1909.03149/) (2019)
- [Establishing Strong Baselines for the New Decade: Sequence Tagging, Syntactic and Semantic Parsing with BERT](https://mlsanity.com/paper/1908.04943/) (2019)
- [Rethinking Why Intermediate-Task Fine-Tuning Works](https://mlsanity.com/paper/2108.11696/) (2021)
- [Second-Order Neural Dependency Parsing with Message Passing and End-to-End Training](https://mlsanity.com/paper/2010.05003/) (2020)
- [Transition-based Semantic Dependency Parsing with Pointer Networks](https://mlsanity.com/paper/2005.13344/) (2020)
- [Robust Incremental Neural Semantic Graph Parsing](https://mlsanity.com/paper/1704.07092/) (2017)
- [Dependency Parsing as MRC-based Span-Span Prediction](https://mlsanity.com/paper/2105.07654/) (2021)
- [Deep Multitask Learning for Semantic Dependency Parsing](https://mlsanity.com/paper/1704.06855/) (2017)

## Abstract (arXiv)

Dependency parsing aims to extract syntactic dependency structure or semantic dependency structure for sentences. Existing methods suffer the drawbacks of lacking universality or highly relying on the auxiliary decoder. To remedy these drawbacks, we propose to achieve universal and schema-free Dependency Parsing (DP) via Sequence Generation (SG) DPSG by utilizing only the pre-trained language model (PLM) without any auxiliary structures or parsing algorithms. We first explore different serialization designing strategies for converting parsing structures into sequences. Then we design dependency units and concatenate these units into the sequence for DPSG. Thanks to the high flexibility of the sequence generation, our DPSG can achieve both syntactic DP and semantic DP using a single model. By concatenating the prefix to indicate the specific schema with the sequence, our DPSG can even accomplish multi-schemata parsing. The effectiveness of our DPSG is demonstrated by the experiments on widely used DP benchmarks, i.e., PTB, CODT, SDP15, and SemEval16. DPSG achieves comparable results with the first-tier methods on all the benchmarks and even the state-of-the-art (SOTA) performance in CODT and SemEval16. This paper demonstrates our DPSG has the potential to be a new parsing paradigm. We will release our codes upon acceptance.