mlsanityData to 14 Oct 2026

PESCO: Prompt-enhanced Self Contrastive Learning for Zero-shot Text Classification

arXiv 2305.14963 ↗First posted 24 May 2023cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

We present PESCO, a novel contrastive learning framework that substantially improves the performance of zero-shot text classification. We formulate text classification as a neural text matching problem where each document is treated as a query, and the system learns the mapping from each query to the relevant class labels by (1) adding prompts to enhance label matching, and (2) using retrieved labels to enrich the training set in a self-training loop of contrastive learning. PESCO achieves state-of-the-art performance on four benchmark text classification datasets. On DBpedia, we achieve 98.5% accuracy without any labeled data, which is close to the fully-supervised result. Extensive experiments and analyses show all the components of PESCO are necessary for improving the performance of zero-shot text classification.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 31 of its 48 references that are in the map. First posted on arXiv 24 May 2023.

Builds on

31 of 48 references.

References · 31

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  7. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  8. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  9. 2020Supervised Contrastive Learningcited by 2,829
  10. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  11. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  12. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
19 more
  1. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  2. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  3. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
  4. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,256
  5. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,127
  6. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 938
  7. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
  8. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 695
  9. 2020Rethinking Pre-training and Self-trainingcited by 352
  10. 2019Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approachcited by 308
  11. 2020CLEAR: Contrastive Learning for Sentence Representationcited by 186
  12. 2020How Context Affects Language Models' Factual Predictionscited by 181
  13. 2020CERT: Contrastive Self-supervised Learning for Language Understandingcited by 169
  14. 2020Pre-training Tasks for Embedding-based Large-scale Retrievalcited by 153
  15. 2022DiffCSE: Difference-based Contrastive Learning for Sentence Embeddingscited by 132
  16. 2021COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretrainingcited by 126
  17. 2020Self-training Improves Pre-training for Natural Language Understandingcited by 96
  18. 2017Task-Oriented Query Reformulation with Reinforcement Learningcited by 39
  19. 2022English Contrastive Learning Can Learn Universal Cross-lingual Sentence Embeddingscited by 26

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 1; 2026: 1.

4 papers.

  1. 2024RulePrompt: Weakly Supervised Text Classification with Prompting PLMs and Self-Iterative Logical Rulescited by 2
  2. 2023Using Large Language Models to Automate Category and Trend Analysis of Scientific Articles: An Application in Ophthalmologycited by 1
  3. 2025Small sample-based adaptive text classification through iterative and contrastive description refinementcited by 0
  4. 2026AutoTail-BSFGM: Class-Balance-Aware Fine-Tuning for Chinese Scholarly Text Classificationcited by 0