# SemEval-2022 Task 2: Multilingual Idiomaticity Detection and Sentence Embedding

arXiv: https://arxiv.org/abs/2204.10050 · HTML: https://mlsanity.com/paper/2204.10050/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 65 references that are in the map.
- First posted on arXiv 21 Apr 2022.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluation](https://mlsanity.com/paper/1708.00055/) (2017)
- [CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data](https://mlsanity.com/paper/1911.00359/) (2019)
- [Efficient Natural Language Response Suggestion for Smart Reply](https://mlsanity.com/paper/1705.00652/) (2017)
- [R-Drop: Regularized Dropout for Neural Networks](https://mlsanity.com/paper/2106.14448/) (2021)
- [Rethinking embedding coupling in pre-trained language models](https://mlsanity.com/paper/2010.12821/) (2020)
- [Still a Pain in the Neck: Evaluating Text Representations on Lexical Composition](https://mlsanity.com/paper/1902.10618/) (2019)
- [AStitchInLanguageModels: Dataset and Methods for the Exploration of Idiomaticity in Pre-Trained Language Models](https://mlsanity.com/paper/2109.04413/) (2021)
- [BERTRAM: Improved Word Embeddings Have Big Impact on Contextualized Model Performance](https://mlsanity.com/paper/1910.07181/) (2019)
- [Idiomatic Expression Identification using Semantic Compatibility](https://mlsanity.com/paper/2110.10064/) (2021)
- [SemEval-2013 Task 4: Free Paraphrases of Noun Compounds](https://mlsanity.com/paper/1911.10421/) (2019)
- [drsphelps at SemEval-2022 Task 2: Learning idiom representations using BERTRAM](https://mlsanity.com/paper/2204.02821/) (2022)
- [HIT at SemEval-2022 Task 2: Pre-trained Language Model for Idioms Detection](https://mlsanity.com/paper/2204.06145/) (2022)
- [JARVix at SemEval-2022 Task 2: It Takes One to Know One? Idiomaticity Detection using Zero and One-Shot Learning](https://mlsanity.com/paper/2202.02394/) (2022)
- [kpfriends at SemEval-2022 Task 2: NEAMER -- Named Entity Augmented Multi-word Expression Recognizer](https://mlsanity.com/paper/2204.08102/) (2022)

## Built on by (most cited first, 30 of 36)

- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [LLMs as a synthesis between symbolic and distributed approaches to language](https://mlsanity.com/paper/2502.11856/) (2025)
- [Sign of the Times: Evaluating the use of Large Language Models for Idiomaticity Detection](https://mlsanity.com/paper/2405.09279/) (2024)
- [A Report on the Euphemisms Detection Shared Task](https://mlsanity.com/paper/2211.13327/) (2022)
- [Code-Mixed Probes Show How Pre-Trained Models Generalise On Code-Switched Text](https://mlsanity.com/paper/2403.04872/) (2024)
- [Enhancing Idiomatic Representation in Multiple Languages via an Adaptive Contrastive Triplet Loss](https://mlsanity.com/paper/2406.15175/) (2024)
- [Construction Artifacts in Metaphor Identification Datasets](https://mlsanity.com/paper/2311.00790/) (2023)
- [PharmaGPT: Domain-Specific Large Language Models for Bio-Pharmaceutical and Chemistry](https://mlsanity.com/paper/2406.18045/) (2024)
- [CoAM: Corpus of All-Type Multiword Expressions](https://mlsanity.com/paper/2412.18151/) (2024)
- [Sample Efficient Approaches for Idiomaticity Detection](https://mlsanity.com/paper/2205.11306/) (2022)
- [Idioms, Probing and Dangerous Things: Towards Structural Probing for Idiomaticity in Vector Space](https://mlsanity.com/paper/2304.14333/) (2023)
- [Multilingual Multi-Figurative Language Detection](https://mlsanity.com/paper/2306.00121/) (2023)
- [Topic Aware Probing: From Sentence Length Prediction to Idiom Identification how reliant are Neural Language Models on Topic?](https://mlsanity.com/paper/2403.02009/) (2024)
- [SemEval-2025 Task 1: AdMIRe -- Advancing Multimodal Idiomaticity Representation](https://mlsanity.com/paper/2503.15358/) (2025)
- [Preferences for Idiomatic Language are Acquired Slowly -- and Forgotten Quickly: A Case Study on Swedish](https://mlsanity.com/paper/2602.03484/) (2026)
- [Effective Cross-Task Transfer Learning for Explainable Natural Language Inference with T5](https://mlsanity.com/paper/2210.17301/) (2022)
- [A Survey of Idiom Datasets for Psycholinguistic and Computational Research](https://mlsanity.com/paper/2508.11828/) (2025)
- [NLP Datasets for Idiom and Figurative Language Tasks](https://mlsanity.com/paper/2511.16345/) (2025)
- [UAlberta at SemEval 2022 Task 2: Leveraging Glosses and Translations for Multilingual Idiomaticity Detection](https://mlsanity.com/paper/2205.14084/) (2022)
- [Is Less More? Quality, Quantity and Context in Idiom Processing with Natural Language Models](https://mlsanity.com/paper/2405.08497/) (2024)
- [Investigating Idiomaticity in Word Representations](https://mlsanity.com/paper/2411.02610/) (2024)
- [A Parallel Cross-Lingual Benchmark for Multimodal Idiomaticity Understanding](https://mlsanity.com/paper/2601.08645/) (2026)
- [SinFoS: A Parallel Dataset for Translating Sinhala Figures of Speech](https://mlsanity.com/paper/2602.09866/) (2026)
- [HiJoNLP at SemEval-2022 Task 2: Detecting Idiomaticity of Multiword Expressions using Multilingual Pretrained Language Models](https://mlsanity.com/paper/2205.13708/) (2022)
- [OCHADAI at SemEval-2022 Task 2: Adversarial Training for Multilingual Idiomaticity Detection](https://mlsanity.com/paper/2206.03025/) (2022)
- [HYU at SemEval-2022 Task 2: Effective Idiomaticity Detection with Consideration at Different Levels of Contextualization](https://mlsanity.com/paper/2206.11854/) (2022)
- [tmn at SemEval-2023 Task 9: Multilingual Tweet Intimacy Detection using XLM-T, Google Translate, and Ensemble Learning](https://mlsanity.com/paper/2304.04054/) (2023)
- [Assessing Linguistic Generalisation in Language Models: A Dataset for Brazilian Portuguese](https://mlsanity.com/paper/2305.14070/) (2023)
- [Unsupervised Paraphrasing of Multiword Expressions](https://mlsanity.com/paper/2306.01443/) (2023)
- [Evaluating Large Language Models on Multiword Expressions in Multilingual and Code-Switched Contexts](https://mlsanity.com/paper/2504.20051/) (2025)

## Abstract (arXiv)

This paper presents the shared task on Multilingual Idiomaticity Detection and Sentence Embedding, which consists of two subtasks: (a) a binary classification task aimed at identifying whether a sentence contains an idiomatic expression, and (b) a task based on semantic text similarity which requires the model to adequately represent potentially idiomatic expressions in context. Each subtask includes different settings regarding the amount of training data. Besides the task description, this paper introduces the datasets in English, Portuguese, and Galician and their annotation procedure, the evaluation metrics, and a summary of the participant systems and their results. The task had close to 100 registered participants organised into twenty five teams making over 650 and 150 submissions in the practice and evaluation phases respectively.