# MTLB-STRUCT @PARSEME 2020: Capturing Unseen Multiword Expressions Using Multi-task Learning and Pre-trained Masked Language Models

arXiv: https://arxiv.org/abs/2011.02541 · HTML: https://mlsanity.com/paper/2011.02541/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Nov 2020.

## Built on by (most cited first, 6 of 6)

- [Romanian Multiword Expression Detection Using Multilingual Adversarial Training and Lateral Inhibition](https://mlsanity.com/paper/2304.11350/) (2023)
- [CoAM: Corpus of All-Type Multiword Expressions](https://mlsanity.com/paper/2412.18151/) (2024)
- [Multilingual Multiword Expression Identification Using Lateral Inhibition and Domain Adaptation](https://mlsanity.com/paper/2306.10419/) (2023)
- [Improving Automatic Hate Speech Detection with Multiword Expression Features](https://mlsanity.com/paper/2106.00237/) (2021)
- [Transformer-based Detection of Multiword Expressions in Flower and Plant Names](https://mlsanity.com/paper/2209.08016/) (2022)
- [IdioLink: Retrieving Meaning Beyond Words Across Idiomatic and Literal Expressions](https://mlsanity.com/paper/2605.22247/) (2026)

## Abstract (arXiv)

This paper describes a semi-supervised system that jointly learns verbal multiword expressions (VMWEs) and dependency parse trees as an auxiliary task. The model benefits from pre-trained multilingual BERT. BERT hidden layers are shared among the two tasks and we introduce an additional linear layer to retrieve VMWE tags. The dependency parse tree prediction is modelled by a linear layer and a bilinear one plus a tree CRF on top of BERT. The system has participated in the open track of the PARSEME shared task 2020 and ranked first in terms of F1-score in identifying unseen VMWEs as well as VMWEs in general, averaged across all 14 languages.