# MICE: Mining Idioms with Contextual Embeddings

arXiv: https://arxiv.org/abs/2008.05759 · HTML: https://mlsanity.com/paper/2008.05759/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Aug 2020.

## Built on by (most cited first, 7 of 7)

- [Semantics of Multiword Expressions in Transformer-Based Models: A Survey](https://mlsanity.com/paper/2401.15393/) (2024)
- [Evaluation of contextual embeddings on less-resourced languages](https://mlsanity.com/paper/2107.10614/) (2021)
- [Getting BART to Ride the Idiomatic Train: Learning to Represent Idiomatic Expressions](https://mlsanity.com/paper/2207.03679/) (2022)
- [Cross-lingual alignments of ELMo contextual embeddings](https://mlsanity.com/paper/2106.15986/) (2021)
- [Unified Representation for Non-compositional and Compositional Expressions](https://mlsanity.com/paper/2310.19127/) (2023)
- [NLP Datasets for Idiom and Figurative Language Tasks](https://mlsanity.com/paper/2511.16345/) (2025)
- [IEKG: A Commonsense Knowledge Graph for Idiomatic Expressions](https://mlsanity.com/paper/2312.06053/) (2023)

## Abstract (arXiv)

Idiomatic expressions can be problematic for natural language processing applications as their meaning cannot be inferred from their constituting words. A lack of successful methodological approaches and sufficiently large datasets prevents the development of machine learning approaches for detecting idioms, especially for expressions that do not occur in the training set. We present an approach, called MICE, that uses contextual embeddings for that purpose. We present a new dataset of multi-word expressions with literal and idiomatic meanings and use it to train a classifier based on two state-of-the-art contextual word embeddings: ELMo and BERT. We show that deep neural networks using both embeddings perform much better than existing approaches, and are capable of detecting idiomatic word use, even for expressions that were not present in the training set. We demonstrate cross-lingual transfer of developed models and analyze the size of the required dataset.