# Parser Training with Heterogeneous Treebanks

arXiv: https://arxiv.org/abs/1805.05089 · HTML: https://mlsanity.com/paper/1805.05089/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 May 2018.

## Built on by (most cited first, 8 of 8)

- [On Generalization in Coreference Resolution](https://mlsanity.com/paper/2109.09667/) (2021)
- [SIP: Injecting a Structural Inductive Bias into a Seq2Seq Model by Simulation](https://mlsanity.com/paper/2310.00796/) (2023)
- [On the Effectiveness of Dataset Embeddings in Mono-lingual,Multi-lingual and Zero-shot Conditions](https://mlsanity.com/paper/2103.01273/) (2021)
- [Bridging Discourse Treebanks with a Unified Rhetorical Structure Parser](https://mlsanity.com/paper/2510.06427/) (2025)
- [Parsing with Pretrained Language Models, Multiple Datasets, and Dataset Embeddings](https://mlsanity.com/paper/2112.03625/) (2021)
- [Weakly Supervised Headline Dependency Parsing](https://mlsanity.com/paper/2301.10371/) (2023)
- [How to Encode Domain Information in Relation Classification](https://mlsanity.com/paper/2404.13760/) (2024)
- [Transfer Learning for an Endangered Slavic Variety: Dependency Parsing in Pomak Across Contact-Shaped Dialects](https://mlsanity.com/paper/2603.28033/) (2026)

## Abstract (arXiv)

How to make the most of multiple heterogeneous treebanks when training a monolingual dependency parser is an open question. We start by investigating previously suggested, but little evaluated, strategies for exploiting multiple treebanks based on concatenating training sets, with or without fine-tuning. We go on to propose a new method based on treebank embeddings. We perform experiments for several languages and show that in many cases fine-tuning and treebank embeddings lead to substantial improvements over single treebanks or concatenation, with average gains of 2.0--3.5 LAS points. We argue that treebank embeddings should be preferred due to their conceptual simplicity, flexibility and extensibility.