mlsanityData to 7 Oct 2026

Boosting Prompt-Based Self-Training With Mapping-Free Automatic Verbalizer for Multi-Class Classification

arXiv 2312.04982 ↗First posted 8 Dec 2023cs.CLcited by 0

On the map: Synthetic data and self-improvement for LLMs → Self-training for few-shot NLP

Abstract (arXiv)

Recently, prompt-based fine-tuning has garnered considerable interest as a core technique for few-shot text classification task. This approach reformulates the fine-tuning objective to align with the Masked Language Modeling (MLM) objective. Leveraging unlabeled data, prompt-based self-training has shown greater effectiveness in binary and three-class classification. However, prompt-based self-training for multi-class classification has not been adequately investigated, despite its significant applicability to real-world scenarios. Moreover, extending current methods to multi-class classification suffers from the verbalizer that extracts the predicted value of manually pre-defined single label word for each class from MLM predictions. Consequently, we introduce a novel, efficient verbalizer structure, named Mapping-free Automatic Verbalizer (MAV). Comprising two fully connected layers, MAV serves as a trainable verbalizer that automatically extracts the requisite word features for classification by capitalizing on all available information from MLM predictions. Experimental results on five multi-class classification datasets indicate MAV's superior self-training efficacy.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Self-training for few-shot NLP, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 31 of its 46 references that are in the map. First posted on arXiv 8 Dec 2023.

Builds on

31 of 46 references.

Closest ancestors · 2

Same direction, newest first

  1. 2021LiST: Lite Prompted Self-training Makes Parameter-Efficient Few-shot Learners12 Oct
  2. 2021Revisiting Self-Training for Few-Shot Learning of Language Model4 Oct

Other references · 29

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  5. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  6. 2018Deep contextualized word representationscited by 2,285
  7. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  8. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  9. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  10. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
  11. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  12. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 800
17 more
  1. 2021FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labelingcited by 537
  2. 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 479
  3. 2021A Survey of Data Augmentation Approaches for NLPcited by 408
  4. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 324
  5. 2021PTR: Prompt Tuning with Rules for Text Classificationcited by 238
  6. 2020MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classificationcited by 186
  7. 2021Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classificationcited by 183
  8. 2020Better Fine-Tuning by Reducing Representational Collapsecited by 156
  9. 2020Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classificationcited by 98
  10. 2021Template-free Prompt Tuning for Few-shot NERcited by 73
  11. 2020A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generationcited by 72
  12. 2022Prototypical Verbalizer for Prompt-based Few-shot Tuningcited by 53
  13. 2022PERFECT: Prompt-free and Efficient Few-shot Learning with Language Modelscited by 40
  14. 2021Reweighting Augmented Samples by Minimizing the Maximal Expected Losscited by 21
  15. 2022Making Pre-trained Language Models End-to-end Few-shot Learners with Contrastive Prompt Tuningcited by 19
  16. 2022Relation Extraction as Open-book Examination: Retrieval-enhanced Prompt Tuningcited by 13
  17. 2022Multi-class Classification from Multiple Unlabeled Datasets with Partial Risk Regularizationcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Unlabeled Data Improves Fine-Grained Image Zero-shot Classification with Multimodal LLMs
  2. 2022Zero-Shot Text Classification with Self-Training
  3. 2023Large Language Models Are Zero-Shot Text Classifiers
  4. 2024Efficient Test-Time Prompt Tuning for Vision-Language Models
  5. 2026TATRA: Training-Free Instance-Adaptive Prompting Through Rephrasing and Aggregation
  6. 2024LLMEmbed: Rethinking Lightweight LLM's Genuine Function in Text Classification
  7. 2023Prompt Algebra for Task Composition
  8. 2025Weighted Multi-Prompt Learning with Description-free Large Language Model Distillation