# Enhancing Idiomatic Representation in Multiple Languages via an Adaptive Contrastive Triplet Loss

arXiv: https://arxiv.org/abs/2406.15175 · HTML: https://mlsanity.com/paper/2406.15175/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 72 references that are in the map.
- First posted on arXiv 21 Jun 2024.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022)
- [Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval](https://mlsanity.com/paper/2007.00808/) (2020)
- [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023)
- [Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models](https://mlsanity.com/paper/2108.08877/) (2021)
- [Large Dual Encoders Are Generalizable Retrievers](https://mlsanity.com/paper/2112.07899/) (2021)
- [Circle Loss: A Unified Perspective of Pair Similarity Optimization](https://mlsanity.com/paper/2002.10857/) (2020)
- [Text and Code Embeddings by Contrastive Pre-Training](https://mlsanity.com/paper/2201.10005/) (2022)
- [DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations](https://mlsanity.com/paper/2006.03659/) (2020)
- [CERT: Contrastive Self-supervised Learning for Language Understanding](https://mlsanity.com/paper/2005.12766/) (2020)
- [Contrastive Learning for Many-to-many Multilingual Neural Machine Translation](https://mlsanity.com/paper/2105.09501/) (2021)
- [Self-Guided Contrastive Learning for BERT Sentence Representations](https://mlsanity.com/paper/2106.07345/) (2021)
- [ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning](https://mlsanity.com/paper/2012.15022/) (2020)
- [SemEval-2022 Task 2: Multilingual Idiomaticity Detection and Sentence Embedding](https://mlsanity.com/paper/2204.10050/) (2022)
- [Is a Single Vector Enough? Exploring Node Polysemy for Network Embedding](https://mlsanity.com/paper/1905.10668/) (2019)
- [PCL: Peer-Contrastive Learning with Diverse Augmentations for Unsupervised Sentence Embeddings](https://mlsanity.com/paper/2201.12093/) (2022)
- [AStitchInLanguageModels: Dataset and Methods for the Exploration of Idiomaticity in Pre-Trained Language Models](https://mlsanity.com/paper/2109.04413/) (2021)
- [Investigating Robustness of Dialog Models to Popular Figurative Language Constructs](https://mlsanity.com/paper/2110.00687/) (2021)
- [Improving Tokenisation by Alternative Treatment of Spaces](https://mlsanity.com/paper/2204.04058/) (2022)
- [SimCSE++: Improving Contrastive Learning for Sentence Embeddings from Two Perspectives](https://mlsanity.com/paper/2305.13192/) (2023)

## Built on by (most cited first, 7 of 7)

- [SemEval-2025 Task 1: AdMIRe -- Advancing Multimodal Idiomaticity Representation](https://mlsanity.com/paper/2503.15358/) (2025)
- [UoR-NCL at SemEval-2025 Task 1: Using Generative LLMs and CLIP Models for Multilingual Multimodal Idiomaticity Representation](https://mlsanity.com/paper/2502.20984/) (2025)
- [NLP Datasets for Idiom and Figurative Language Tasks](https://mlsanity.com/paper/2511.16345/) (2025)
- [From Input Perception to Predictive Insight: Modeling Model Blind Spots Before They Become Errors](https://mlsanity.com/paper/2509.20065/) (2025)
- [A Rising Tide Lifts All Boats: MTQE Rewards for Idioms Improve General Translation Quality](https://mlsanity.com/paper/2601.06307/) (2026)
- [Span Modeling for Idiomaticity and Figurative Language Detection with Span Contrastive Loss](https://mlsanity.com/paper/2603.22799/) (2026)
- [IdioLink: Retrieving Meaning Beyond Words Across Idiomatic and Literal Expressions](https://mlsanity.com/paper/2605.22247/) (2026)

## Abstract (arXiv)

Accurately modeling idiomatic or non-compositional language has been a longstanding challenge in Natural Language Processing (NLP). This is partly because these expressions do not derive their meanings solely from their constituent words, but also due to the scarcity of relevant data resources, and their impact on the performance of downstream tasks such as machine translation and simplification. In this paper we propose an approach to model idiomaticity effectively using a triplet loss that incorporates the asymmetric contribution of components words to an idiomatic meaning for training language models by using adaptive contrastive learning and resampling miners to build an idiomatic-aware learning objective. Our proposed method is evaluated on a SemEval challenge and outperforms previous alternatives significantly in many metrics.