mlsanityData to 7 Oct 2026

ContrastNER: Contrastive-based Prompt Tuning for Few-shot NER

arXiv 2305.17951 ↗First posted 29 May 2023cs.CL cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Prompt-based language models have produced encouraging results in numerous applications, including Named Entity Recognition (NER) tasks. NER aims to identify entities in a sentence and provide their types. However, the strong performance of most available NER approaches is heavily dependent on the design of discrete prompts and a verbalizer to map the model-predicted outputs to entity categories, which are complicated undertakings. To address these challenges, we present ContrastNER, a prompt-based NER framework that employs both discrete and continuous tokens in prompts and uses a contrastive learning approach to learn the continuous prompts and forecast entity types. The experimental results demonstrate that ContrastNER obtains competitive performance to the state-of-the-art NER methods in high-resource settings and outperforms the state-of-the-art models in low-resource circumstances without requiring extensive manual prompt engineering and verbalizer design.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 44 of its 57 references that are in the map. First posted on arXiv 29 May 2023.

Builds on

44 of 57 references.

References · 44

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  8. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  9. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  10. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  11. 2020Supervised Contrastive Learningcited by 2,803
  12. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
32 more
  1. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  2. 2019Language Models as Knowledge Bases?cited by 1,922
  3. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  4. 2018Deep Graph Infomaxcited by 1,101
  5. 2019How Can We Know What Language Models Know?cited by 953
  6. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  7. 2021GPT Understands, Toocited by 715
  8. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 686
  9. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
  10. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  11. 2015An Introduction to Convolutional Neural Networkscited by 510
  12. 2016End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRFcited by 341
  13. 2020Keystroke Dynamics as Part of Lifeloggingcited by 334
  14. 2020LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attentioncited by 326
  15. 2016Large-Margin Softmax Loss for Convolutional Neural Networkscited by 296
  16. 2020Supervised Contrastive Learning for Pre-trained Language Model Fine-tuningcited by 275
  17. 2015Named Entity Recognition with Bidirectional LSTM-CNNscited by 201
  18. 2021How Many Data Points is a Prompt Worth?cited by 197
  19. 2021Template-Based Named Entity Recognition Using BARTcited by 153
  20. 2021Few-NERD: A Few-Shot Named Entity Recognition Datasetcited by 150
  21. 2021A Unified Generative Framework for Various NER Subtaskscited by 141
  22. 2018Few-shot classification in Named Entity Recognition Taskcited by 64
  23. 2020Pretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language Modelscited by 49
  24. 2017Context-aware Captions from Context-agnostic Supervisioncited by 43
  25. 2021Prototypical Representation Learning for Relation Extractioncited by 33
  26. 2019Hierarchical Contextualized Representation for Named Entity Recognitioncited by 32
  27. 2018Adversarial Contrastive Estimationcited by 23
  28. 2020Example-Based Named Entity Recognitioncited by 19
  29. 2022Making Pre-trained Language Models End-to-end Few-shot Learners with Contrastive Prompt Tuningcited by 19
  30. 2018Deep contextualized word representations for detecting sarcasm and ironycited by 18
  31. 2019GRN: Gated Relation Network to Enhance Convolutional Neural Network for Named Entity Recognitioncited by 18
  32. 2022QaNER: Prompting Question Answering Models for Few-shot Named Entity Recognitioncited by 15

Built on by

Cited by year

Citing papers per year: 2024: 3.

3 papers.

  1. 2024Natural Language Processing for the Legal Domain: A Survey of Tasks, Datasets, Models, and Challengescited by 42
  2. 2024Few-shot clinical entity recognition in English, French and Spanish: masked language models outperform generative model promptingcited by 12
  3. 2024Large-Scale Label Interpretation Learning for Few-Shot Named Entity Recognitioncited by 2