mlsanityData to 7 Oct 2026

CONTaiNER: Few-Shot Named Entity Recognition via Contrastive Learning

arXiv 2109.07589 ↗First posted 15 Sep 2021cs.CLcited by 64

Not on the map. Why?

Abstract (arXiv)

Named Entity Recognition (NER) in Few-Shot setting is imperative for entity tagging in low resource domains. Existing approaches only learn class-specific semantic features and intermediate representations from source domains. This affects generalizability to unseen target domains, resulting in suboptimal performances. To this end, we present CONTaiNER, a novel contrastive learning technique that optimizes the inter-token distribution distance for Few-Shot NER. Instead of optimizing class-specific attributes, CONTaiNER optimizes a generalized objective of differentiating between token categories based on their Gaussian-distributed embeddings. This effectively alleviates overfitting issues originating from training domains. Our experiments in several traditional test domains (OntoNotes, CoNLL'03, WNUT '17, GUM) and a new large scale Few-Shot NER dataset (Few-NERD) demonstrate that on average, CONTaiNER outperforms previous methods by 3%-13% absolute F1 points while showing consistent performance trends, even in challenging scenarios where previous approaches could not achieve appreciable performance.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). Builds on 29 of its 52 references that are in the map. First posted on arXiv 15 Sep 2021.

Builds on

29 of 52 references.

References · 29

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  4. 2020Supervised Contrastive Learningcited by 2,803
  5. 2018Deep contextualized word representationscited by 2,285
  6. 2016Matching Networks for One Shot Learningcited by 2,246
  7. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  8. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 806
  9. 2016Neural Architectures for Named Entity Recognitioncited by 548
  10. 2015Bidirectional LSTM-CRF Models for Sequence Taggingcited by 544
  11. 2021True Few-Shot Learning with Language Modelscited by 343
  12. 2016End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRFcited by 341
17 more
  1. 2017Deep Gaussian Embedding of Graphs: Unsupervised Inductive Learning via Rankingcited by 332
  2. 2014Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networkscited by 299
  3. 2018FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluationcited by 254
  4. 2019Unsupervised Embedding Learning via Invariant and Spreading Instance Featurecited by 231
  5. 2019Local Aggregation for Unsupervised Learning of Visual Embeddingscited by 204
  6. 2017Multi-task Self-Supervised Visual Learningcited by 200
  7. 2019SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learningcited by 194
  8. 2021Template-Based Named Entity Recognition Using BARTcited by 153
  9. 2021Few-NERD: A Few-Shot Named Entity Recognition Datasetcited by 150
  10. 2014Word Representations via Gaussian Embeddingcited by 96
  11. 2020Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Networkcited by 83
  12. 2018Few-shot classification in Named Entity Recognition Taskcited by 64
  13. 2019Induction Networks for Few-Shot Text Classificationcited by 64
  14. 2019Few-shot Text Classification with Distributional Signaturescited by 58
  15. 2020Few-Shot Named Entity Recognition: A Comprehensive Studycited by 41
  16. 2019Label-Agnostic Sequence Labeling by Copying Nearest Neighborscited by 27
  17. 2018Hierarchical Density Order Embeddingscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 22; 2023: 28; 2024: 8; 2025: 1; 2026: 4.

64 papers.

Sort:

By area:

  1. 2022A Contrastive Framework for Neural Text Generationcited by 224
  2. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
  3. 2022Language Models Can See: Plugging Visual Controls in Text Generationcited by 89
  4. 2023A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4cited by 81
  5. 2023PromptNER: Prompting For Named Entity Recognitioncited by 35
  6. 2021TaCL: Improving BERT Pre-training with Token-aware Contrastive Learningcited by 32
  7. 2022Modality-Aware Contrastive Instance Learning with Self-Distillation for Weakly-Supervised Audio-Visual Violence Detectioncited by 26
  8. 2023Mask-guided BERT for Few Shot Text Classificationcited by 25
  9. 2022Optimizing Bi-Encoder for Named Entity Recognition via Contrastive Learningcited by 24
  10. 2022Decomposed Meta-Learning for Few-Shot Named Entity Recognitioncited by 22
  11. 2023Prompt Tuning Pushes Farther, Contrastive Learning Pulls Closer: A Two-Stage Approach to Mitigate Social Biasescited by 21
  12. 2023Unified Low-Resource Sequence Labeling by Sample-Aware Dynamic Sparse Finetuningcited by 21
38 more
  1. 2022Improving Word Translation via Two-Stage Contrastive Learningcited by 18
  2. 2022Few-shot Named Entity Recognition with Entity-level Prototypical Network Enhanced by Dispersedly Distributed Prototypescited by 17
  3. 2022Few-shot Named Entity Recognition with Self-describing Networkscited by 16
  4. 2022Improving the Sample Efficiency of Prompt Tuning with Domain Adaptationcited by 15
  5. 2023How far is Language Model from 100% Few-shot Named Entity Recognition in Medical Domaincited by 14
  6. 2023mCL-NER: Cross-Lingual Named Entity Recognition via Multi-view Contrastive Learningcited by 14
  7. 2022Prompt-Based Metric Learning for Few-Shot NERcited by 12
  8. 2022KRACL: Contrastive Learning with Graph Context Modeling for Sparse Knowledge Graph Completioncited by 10
  9. 2022Distilling Causal Effect from Miscellaneous Other-Class for Continual Named Entity Recognitioncited by 10
  10. 2022Few-Shot Document-Level Event Argument Extractioncited by 9
  11. 2023VicunaNER: Zero/Few-shot Named Entity Recognition using Vicunacited by 9
  12. 2023A Multi-Task Semantic Decomposition Framework with Task-specific Pre-training for Few-Shot NERcited by 9
  13. 2022SpanProto: A Two-stage Span-based Prototypical Network for Few-shot Named Entity Recognitioncited by 8
  14. 2023Improving Self-training for Cross-lingual Named Entity Recognition with Contrastive and Prototype Learningcited by 8
  15. 2022FactMix: Using a Few Labeled In-domain Examples to Generalize to Cross-domain Named Entity Recognitioncited by 7
  16. 2023Learning In-context Learning for Named Entity Recognitioncited by 7
  17. 2023A Prototypical Semantic Decoupling Method via Joint Contrastive Learning for Few-Shot Name Entity Recognitioncited by 6
  18. 2024Text2MDT: Extracting Medical Decision Trees from Medical Textscited by 6
  19. 2023Medical Question Summarization with Entity-driven Contrastive Learningcited by 5
  20. 2023Few-shot Event Detection: An Empirical Study and a Unified Viewcited by 5
  21. 2023Type-Aware Decomposed Framework for Few-Shot Named Entity Recognitioncited by 4
  22. 2023PromptNER: A Prompting Method for Few-shot Named Entity Recognition via k Nearest Neighbor Searchcited by 4
  23. 2023W-procer: Weighted Prototypical Contrastive Learning for Medical Few-Shot Named Entity Recognitioncited by 4
  24. 2023Robust Few-Shot Named Entity Recognition with Boundary Discrimination and Correlation Purificationcited by 4
  25. 2023Uncertainty-aware Self-training for Low-resource Neural Sequence Labelingcited by 3
  26. 2024Fighting Against the Repetitive Training and Sample Dependency Problem in Few-shot Named Entity Recognitioncited by 3
  27. 2022Block-SCL: Blocking Matters for Supervised Contrastive Learning in Product Matchingcited by 2
  28. 2022Few-shot Classification with Hypersphere Modeling of Prototypescited by 2
  29. 2022Few-Shot Nested Named Entity Recognitioncited by 2
  30. 2023TKDP: Threefold Knowledge-enriched Deep Prompt Tuning for Few-shot Named Entity Recognitioncited by 2
  31. 2024Large-Scale Label Interpretation Learning for Few-Shot Named Entity Recognitioncited by 2
  32. 2022A Comparative Study of Pre-trained Encoders for Low-Resource Named Entity Recognitioncited by 1
  33. 2022Reconciliation of Pre-trained Models and Prototypical Neural Networks in Few-shot Named Entity Recognitioncited by 1
  34. 2023Meta-Learning Triplet Network with Adaptive Margins for Few-Shot Named Entity Recognitioncited by 1
  35. 2023Dialogue State Distillation Network with Inter-slot Contrastive Learning for Dialogue State Trackingcited by 1
  36. 2023Out-of-Distribution Generalization in Text Classification: Past, Present, and Futurecited by 1
  37. 2023Unifying Token and Span Level Supervisions for Few-Shot Sequence Labelingcited by 1
  38. 2023HierarchicalContrast: A Coarse-to-Fine Contrastive Learning Framework for Cross-Domain Zero-Shot Slot Fillingcited by 1

50 most cited of 64 All 64 (JSON)