# HiJoNLP at SemEval-2022 Task 2: Detecting Idiomaticity of Multiword Expressions using Multilingual Pretrained Language Models

arXiv: https://arxiv.org/abs/2205.13708 · HTML: https://mlsanity.com/paper/2205.13708/

## Key facts

- No citing papers in the map yet (data through 7 Oct 2026).
- Builds on 9 of its 19 references that are in the map.
- First posted on arXiv 27 May 2022.

## Builds on (most cited first, 9 of 9)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [Assessing Phrasal Representation and Composition in Transformers](https://mlsanity.com/paper/2010.03763/) (2020)
- [A Challenge Set Approach to Evaluating Machine Translation](https://mlsanity.com/paper/1704.07431/) (2017)
- [SemEval-2022 Task 2: Multilingual Idiomaticity Detection and Sentence Embedding](https://mlsanity.com/paper/2204.10050/) (2022)
- [Still a Pain in the Neck: Evaluating Text Representations on Lexical Composition](https://mlsanity.com/paper/1902.10618/) (2019)
- [AStitchInLanguageModels: Dataset and Methods for the Exploration of Idiomaticity in Pre-Trained Language Models](https://mlsanity.com/paper/2109.04413/) (2021)
- [Classifying Idiomatic and Literal Expressions Using Topic Models and Intensity of Emotions](https://mlsanity.com/paper/1802.09961/) (2018)

## Abstract (arXiv)

This paper describes an approach to detect idiomaticity only from the contextualized representation of a MWE over multilingual pretrained language models. Our experiments find that larger models are usually more effective in idiomaticity detection. However, using a higher layer of the model may not guarantee a better performance. In multilingual scenarios, the convergence of different languages are not consistent and rich-resource languages have big advantages over other languages.