# Sample Efficient Approaches for Idiomaticity Detection

arXiv: https://arxiv.org/abs/2205.11306 · HTML: https://mlsanity.com/paper/2205.11306/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 21 references that are in the map.
- First posted on arXiv 23 May 2022.

## Builds on (most cited first, 9 of 9)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [Assessing Phrasal Representation and Composition in Transformers](https://mlsanity.com/paper/2010.03763/) (2020)
- [SemEval-2022 Task 2: Multilingual Idiomaticity Detection and Sentence Embedding](https://mlsanity.com/paper/2204.10050/) (2022)
- [Still a Pain in the Neck: Evaluating Text Representations on Lexical Composition](https://mlsanity.com/paper/1902.10618/) (2019)
- [AStitchInLanguageModels: Dataset and Methods for the Exploration of Idiomaticity in Pre-Trained Language Models](https://mlsanity.com/paper/2109.04413/) (2021)
- [BERTRAM: Improved Word Embeddings Have Big Impact on Contextualized Model Performance](https://mlsanity.com/paper/1910.07181/) (2019)
- [Bertinho: Galician BERT Representations](https://mlsanity.com/paper/2103.13799/) (2021)
- [drsphelps at SemEval-2022 Task 2: Learning idiom representations using BERTRAM](https://mlsanity.com/paper/2204.02821/) (2022)

## Built on by (most cited first, 3 of 3)

- [Investigating Idiomaticity in Word Representations](https://mlsanity.com/paper/2411.02610/) (2024)
- [PolyFrame at MWE-2026 AdMIRe 2: When Words Are Not Enough: Multimodal Idiom Disambiguation](https://mlsanity.com/paper/2602.18652/) (2026)
- [To What Extent Do Large Language Models Understand Bangla Idioms?](https://mlsanity.com/paper/2609.03410/) (2026)

## Abstract (arXiv)

Deep neural models, in particular Transformer-based pre-trained language models, require a significant amount of data to train. This need for data tends to lead to problems when dealing with idiomatic multiword expressions (MWEs), which are inherently less frequent in natural text. As such, this work explores sample efficient methods of idiomaticity detection. In particular we study the impact of Pattern Exploit Training (PET), a few-shot method of classification, and BERTRAM, an efficient method of creating contextual embeddings, on the task of idiomaticity detection. In addition, to further explore generalisability, we focus on the identification of MWEs not present in the training data. Our experiments show that while these methods improve performance on English, they are much less effective on Portuguese and Galician, leading to an overall performance about on par with vanilla mBERT. Regardless, we believe sample efficient methods for both identifying and representing potentially idiomatic MWEs are very encouraging and hold significant potential for future exploration.