# Binary Token-Level Classification with DeBERTa for All-Type MWE Identification: A Lightweight Approach with Linguistic Enhancement

arXiv: https://arxiv.org/abs/2601.19360 · HTML: https://mlsanity.com/paper/2601.19360/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 25 references that are in the map.
- First posted on arXiv 27 Jan 2026.

## Builds on (most cited first, 5 of 5)

- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [Span-based Joint Entity and Relation Extraction with Transformer Pre-training](https://mlsanity.com/paper/1909.07755/) (2019)
- [Comprehensive Supersense Disambiguation of English Prepositions and Possessives](https://mlsanity.com/paper/1805.04905/) (2018)
- [Bridging the Gap: Attending to Discontinuity in Identification of Multiword Expressions](https://mlsanity.com/paper/1902.10667/) (2019)
- [CoAM: Corpus of All-Type Multiword Expressions](https://mlsanity.com/paper/2412.18151/) (2024)

## Built on by (most cited first, 1 of 1)

- [From Lemmas to Dependencies: What Signals Drive Light Verbs Classification?](https://mlsanity.com/paper/2602.04127/) (2026)

## Abstract (arXiv)

We present a comprehensive approach for multiword expression (MWE) identification that combines binary token-level classification, linguistic feature integration, and data augmentation. Our DeBERTa-v3-large model achieves 69.8% F1 on the CoAM dataset, surpassing the best results (Qwen-72B, 57.8% F1) on this dataset by 12 points while using 165x fewer parameters. We achieve this performance by (1) reformulating detection as binary token-level START/END/INSIDE classification rather than span-based prediction, (2) incorporating NP chunking and dependency features that help discontinuous and NOUN-type MWEs identification, and (3) applying oversampling that addresses severe class imbalance in the training data. We confirm the generalization of our method on the STREUSLE dataset, achieving 78.9% F1. These results demonstrate that carefully designed smaller models can substantially outperform LLMs on structured NLP tasks, with important implications for resource-constrained deployments.