mlsanityData to 14 Oct 2026

Labels Need Prompts Too: Mask Matching for Natural Language Understanding Tasks

arXiv 2312.08726 ↗First posted 14 Dec 2023cs.CL cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Textual label names (descriptions) are typically semantically rich in many natural language understanding (NLU) tasks. In this paper, we incorporate the prompting methodology, which is widely used to enrich model input, into the label side for the first time. Specifically, we propose a Mask Matching method, which equips an input with a prompt and its label with another, and then makes predictions by matching their mask representations. We evaluate our method extensively on 8 NLU tasks with 14 datasets. The experimental results show that Mask Matching significantly outperforms its counterparts of fine-tuning and conventional prompt-tuning, setting up state-of-the-art performances in several datasets. Mask Matching is particularly good at handling NLU tasks with large label counts and informative label names. As pioneering efforts that investigate the label-side prompt, we also discuss open issues for future study.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 42 of its 59 references that are in the map. First posted on arXiv 14 Dec 2023.

Builds on

42 of 59 references.

References · 42

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  7. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  8. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  9. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,492
  10. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,256
  11. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
  12. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 938
30 more
  1. 2018WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representationscited by 372
  2. 2019Matching the Blanks: Distributional Similarity for Relation Learningcited by 272
  3. 2021PTR: Prompt Tuning with Rules for Text Classificationcited by 238
  4. 2022Unified Structure Generation for Universal Information Extractioncited by 221
  5. 2021KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extractioncited by 180
  6. 2021Few-NERD: A Few-Shot Named Entity Recognition Datasetcited by 152
  7. 2021Entailment as Few-Shot Learnercited by 123
  8. 2020TACRED Revisited: A Thorough Evaluation of the TACRED Relation Extraction Taskcited by 96
  9. 2021BertGCN: Transductive Text Classification by Combining GCN and BERTcited by 74
  10. 2021Label Verbalization and Entailment for Effective Zero- and Few-Shot Relation Extractioncited by 73
  11. 2021Prompt-Learning for Fine-Grained Entity Typingcited by 62
  12. 2022PromDA: Prompt-based Data Augmentation for Low-Resource NLU Taskscited by 51
  13. 2022Problems with Cosine as a Measure of Embedding Similarity for High Frequency Wordscited by 49
  14. 2021Good Examples Make A Faster Learner: Simple Demonstration-based Learning for Low-resource NERcited by 38
  15. 2020ERNIE-Doc: A Retrospective Long-Document Modeling Transformercited by 31
  16. 2022HPT: Hierarchy-aware Prompt Tuning for Hierarchical Text Classificationcited by 31
  17. 2022METRO: Efficient Denoising Pretraining of Large Scale Autoencoding Language Models with Model Generated Signalscited by 20
  18. 2021Improving Paraphrase Detection with the Adversarial Paraphrasing Taskcited by 19
  19. 2021MirrorWiC: On Eliciting Word-in-Context Representations from Pretrained Language Modelscited by 18
  20. 2022Unified Semantic Typing with Meaningful Label Inferencecited by 18
  21. 2022Text Smoothing: Enhance Various Data Augmentation Methods on Text Classification Taskscited by 13
  22. 2022Natural Language Inference Prompts for Zero-shot Emotion Classification in Text across Corporacited by 13
  23. 2022Reviewing Labels: Label Graph Network with Top-k Prediction Set for Relation Extractioncited by 10
  24. 2022Few-Shot Stance Detection via Target-Aware Prompt Distillationcited by 9
  25. 2022Few-Shot Fine-Grained Entity Typing with Automatic Label Interpretation and Instance Generationcited by 9
  26. 2022Sequence Generation with Label Augmentation for Relation Extractioncited by 9
  27. 2022Progressive Class Semantic Matching for Semi-supervised Text Classificationcited by 7
  28. 2021Marked Attribute Bias in Natural Language Inferencecited by 6
  29. 2022Improving Task Generalization via Unified Schema Promptcited by 6
  30. 2022Ered: Enhanced Text Representations with Entities and Descriptionscited by 1

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025MPL: Multiple Programming Languages with Large Language Models for Information Extractioncited by 2