# Towards Better UD Parsing: Deep Contextualized Word Embeddings, Ensemble, and Treebank Concatenation

arXiv: https://arxiv.org/abs/1807.03121 · HTML: https://mlsanity.com/paper/1807.03121/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jul 2018.

## Built on by (most cited first, 21 of 21)

- [Large-Scale Contextualised Language Modelling for Norwegian](https://mlsanity.com/paper/2104.06546/) (2021)
- [DAMO-NLP at SemEval-2022 Task 11: A Knowledge-based System for Multilingual Named Entity Recognition](https://mlsanity.com/paper/2203.00545/) (2022)
- [Article Reranking by Memory-Enhanced Key Sentence Matching for Detecting Previously Fact-Checked Claims](https://mlsanity.com/paper/2112.10322/) (2021)
- [Meta-Learning for Fast Cross-Lingual Adaptation in Dependency Parsing](https://mlsanity.com/paper/2104.04736/) (2021)
- [!Qu'e maravilla! Multimodal Sarcasm Detection in Spanish: a Dataset and a Baseline](https://mlsanity.com/paper/2105.05542/) (2021)
- [Resources for Turkish Natural Language Processing: A critical survey](https://mlsanity.com/paper/2204.05042/) (2022)
- [Evaluation Benchmarks for Spanish Sentence Representations](https://mlsanity.com/paper/2204.07571/) (2022)
- [Evaluation of contextual embeddings on less-resourced languages](https://mlsanity.com/paper/2107.10614/) (2021)
- [Neural Supervised Domain Adaptation by Augmenting Pre-trained Models with Random Units](https://mlsanity.com/paper/2106.04935/) (2021)
- [A Comprehensive Analysis of Static Word Embeddings for Turkish](https://mlsanity.com/paper/2405.07778/) (2024)
- [Revisiting Tri-training of Dependency Parsers](https://mlsanity.com/paper/2109.08122/) (2021)
- [Bridging Pre-trained Language Models and Hand-crafted Features for Unsupervised POS Tagging](https://mlsanity.com/paper/2203.10315/) (2022)
- [Android Security using NLP Techniques: A Review](https://mlsanity.com/paper/2107.03072/) (2021)
- [Multimodal Approach for Metadata Extraction from German Scientific Publications](https://mlsanity.com/paper/2111.05736/) (2021)
- [Exploring Descriptions of Movement Through Geovisual Analytics](https://mlsanity.com/paper/2204.09588/) (2022)
- [Zero-shot Cross-Linguistic Learning of Event Semantics](https://mlsanity.com/paper/2207.02356/) (2022)
- [The Fragility of Multi-Treebank Parsing Evaluation](https://mlsanity.com/paper/2209.06699/) (2022)
- [Empirical Analysis for Unsupervised Universal Dependency Parse Tree Aggregation](https://mlsanity.com/paper/2403.19183/) (2024)
- [Fake Advertisements Detection Using Automated Multimodal Learning: A Case Study for Vietnamese Real Estate Data](https://mlsanity.com/paper/2501.10848/) (2025)
- [Scraping the Shadows: Deep Learning Breakthroughs in Dark Web Intelligence](https://mlsanity.com/paper/2504.02872/) (2025)
- [Tokenization and Morphological Fidelity in Uralic NLP: A Cross-Lingual Evaluation](https://mlsanity.com/paper/2602.04241/) (2026)

## Abstract (arXiv)

This paper describes our system (HIT-SCIR) submitted to the CoNLL 2018 shared task on Multilingual Parsing from Raw Text to Universal Dependencies. We base our submission on Stanford's winning system for the CoNLL 2017 shared task and make two effective extensions: 1) incorporating deep contextualized word embeddings into both the part of speech tagger and parser; 2) ensembling parsers trained with different initialization. We also explore different ways of concatenating treebanks for further improvements. Experimental results on the development data show the effectiveness of our methods. In the final evaluation, our system was ranked first according to LAS (75.84%) and outperformed the other systems by a large margin.