# Multi-Task Learning by using Contextualized Word Representations for Syntactic Parsing of a Morphologically Rich Language

arXiv: https://arxiv.org/abs/2609.29855 · HTML: https://mlsanity.com/paper/2609.29855/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 78 references that are in the map.
- First posted on arXiv 24 Sep 2026.

## Builds on (most cited first, 17 of 17)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [Deep Biaffine Attention for Neural Dependency Parsing](https://mlsanity.com/paper/1611.01734/) (2016)
- [Constituency Parsing with a Self-Attentive Encoder](https://mlsanity.com/paper/1805.01052/) (2018)
- [Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations](https://mlsanity.com/paper/1603.04351/) (2016)
- [A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methods](https://mlsanity.com/paper/2204.03508/) (2022)
- [Stack-Pointer Networks for Dependency Parsing](https://mlsanity.com/paper/1805.01087/) (2018)
- [Viable Dependency Parsing as Sequence Labeling](https://mlsanity.com/paper/1902.10505/) (2019)
- [Constituent Parsing as Sequence Labeling](https://mlsanity.com/paper/1810.08994/) (2018)
- [Multitask Pointer Network for Multi-Representational Parsing](https://mlsanity.com/paper/2009.09730/) (2020)
- [Sequence Labeling Parsing by Learning Across Representations](https://mlsanity.com/paper/1907.01339/) (2019)
- [UQA: Corpus for Urdu Question Answering](https://mlsanity.com/paper/2405.01458/) (2024)
- [WER We Stand: Benchmarking Urdu ASR Models](https://mlsanity.com/paper/2409.11252/) (2024)
- [Urdu Word Segmentation using Conditional Random Fields (CRFs)](https://mlsanity.com/paper/1806.05432/) (2018)
- [Multi-Task Learning for Joint Semantic Role and Proto-Role Labeling](https://mlsanity.com/paper/2210.07270/) (2022)
- [UrduLLaMA 1.0: Dataset Curation, Preprocessing, and Evaluation in Low-Resource Settings](https://mlsanity.com/paper/2502.16961/) (2025)

## Abstract (arXiv)

We address the challenge of syntactic parsing for Urdu, a morphologically rich language, and present state-of-the-art results for both constituency and dependency parsing. This paper offers four major contributions: 1) the conversion of the CLE-UTB phrase structure treebank into a dependency treebank by developing language-specific head-word and phrase-to-dependency label mapping rules; 2) a novel sequence labeling scheme that transforms the parsing task into a unified representation; 3) the training of contextualized word representations on a large 220 million tokens Urdu corpus collected from the web; and 4) development of parsing framework using two learning paradigms, single-task and multi-task learning. Several post-processing rules are applied to improve the quality of the automatically converted dependency structure treebank. The proposed sequence labeling scheme enables the use of a shared architecture that learns the syntactic structures from both grammatical structures simultaneously and hence improves generalization. Experiments show that the multi-task learning setup significantly enhances parsing performance, achieving an F1 score of 91.39 for constituency parsing (an improvement of 3.29 points) and a labeled attachment score of 85.69 for dependency parsing (an improvement of 1.49 points). These results demonstrate that learning cross-task representations provides measurable benefits and advances the state of syntactic parsing for Urdu.