# An Investigation of the Interactions Between Pre-Trained Word Embeddings, Character Models and POS Tags in Dependency Parsing

arXiv: https://arxiv.org/abs/1808.09060 · HTML: https://mlsanity.com/paper/1808.09060/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Aug 2018.

## Built on by (most cited first, 10 of 10)

- [Does Your Model Classify Entities Reasonably? Diagnosing and Mitigating Spurious Correlations in Entity Typing](https://mlsanity.com/paper/2205.12640/) (2022)
- [A Modest Pareto Optimisation Analysis of Dependency Parsers in 2021](https://mlsanity.com/paper/2106.04216/) (2021)
- [A Falta de Pan, Buenas Son Tortas: The Efficacy of Predicted UPOS Tags for Low Resource UD Parsing](https://mlsanity.com/paper/2106.04222/) (2021)
- [Instance-Based Neural Dependency Parsing](https://mlsanity.com/paper/2109.13497/) (2021)
- [What Taggers Fail to Learn, Parsers Need the Most](https://mlsanity.com/paper/2104.01083/) (2021)
- [Sort by Structure: Language Model Ranking as Dependency Probing](https://mlsanity.com/paper/2206.04935/) (2022)
- [Parsing linearizations appreciate PoS tags - but some are fussy about errors](https://mlsanity.com/paper/2210.15219/) (2022)
- [Tokenization Strategies for Low-Resource Agglutinative Languages in Word2Vec: Case Study on Turkish and Finnish](https://mlsanity.com/paper/2509.14238/) (2025)
- [The Fragility of Multi-Treebank Parsing Evaluation](https://mlsanity.com/paper/2209.06699/) (2022)
- [Another Dead End for Morphological Tags? Perturbed Inputs and Parsing](https://mlsanity.com/paper/2305.15119/) (2023)

## Abstract (arXiv)

We provide a comprehensive analysis of the interactions between pre-trained word embeddings, character models and POS tags in a transition-based dependency parser. While previous studies have shown POS information to be less important in the presence of character models, we show that in fact there are complex interactions between all three techniques. In isolation each produces large improvements over a baseline system using randomly initialised word embeddings only, but combining them quickly leads to diminishing returns. We categorise words by frequency, POS tag and language in order to systematically investigate how each of the techniques affects parsing quality. For many word categories, applying any two of the three techniques is almost as good as the full combined system. Character models tend to be more important for low-frequency open-class words, especially in morphologically rich languages, while POS tags can help disambiguate high-frequency function words. We also show that large character embedding sizes help even for languages with small character sets, especially in morphologically rich languages.