PolyFrame at MWE-2026 AdMIRe 2: When Words Are Not Enough: Multimodal Idiom Disambiguation
Not on the map. Why?
Abstract (arXiv)
Multimodal models struggle with idiomatic expressions due to their non-compositional meanings, a challenge amplified in multilingual settings. We introduced PolyFrame, our system for the MWE-2026 AdMIRe2 shared task on multimodal idiom disambiguation, featuring a unified pipeline for both image+text ranking (Subtask A) and text-only caption ranking (Subtask B). All model variants retain frozen CLIP-style vision--language encoders and the multilingual BGE M3 encoder, training only lightweight modules: a logistic regression and LLM-based sentence-type predictor, idiom synonym substitution, distractor-aware scoring, and Borda rank fusion. Starting from a CLIP baseline (26.7% Top-1 on English dev, 6.7% on English test), adding idiom-aware paraphrasing and explicit sentence-type classification increased performance to 60.0% Top-1 on English and 60.0% Top-1 (0.822 NDCG@5) in zero-shot transfer to Portuguese. On the multilingual blind test, our systems achieved average Top-1/NDCG scores of 0.35/0.73 for Subtask A and 0.32/0.71 for Subtask B across 15 languages. Ablation results highlight idiom-aware rewriting as the main contributor to performance, while sentence-type prediction and multimodal fusion enhance robustness. These findings suggest that effective idiom disambiguation is feasible without fine-tuning large multimodal encoders.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 11 of its 20 references that are in the map. First posted on arXiv 20 Feb 2026.
Builds on
11 of 20 references.
References · 11
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
- 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,073
- 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,026
- 2024Sign of the Times: Evaluating the use of Large Language Models for Idiomaticity Detectioncited by 11
- 2022The Devil is in the Details: On the Pitfalls of Vocabulary Selection in Neural Machine Translationcited by 3
- 2022Sample Efficient Approaches for Idiomaticity Detectioncited by 3
- 2025SemEval-2025 Task 1: AdMIRe -- Advancing Multimodal Idiomaticity Representationcited by 3
- 2025UoR-NCL at SemEval-2025 Task 1: Using Generative LLMs and CLIP Models for Multilingual Multimodal Idiomaticity Representationcited by 2
- 2025A Survey of Idiom Datasets for Psycholinguistic and Computational Researchcited by 2
- 2026A Parallel Cross-Lingual Benchmark for Multimodal Idiomaticity Understandingcited by 1
Built on by
Not cited yet.