# TKDP: Threefold Knowledge-enriched Deep Prompt Tuning for Few-shot Named Entity Recognition

arXiv: https://arxiv.org/abs/2306.03974 · HTML: https://mlsanity.com/paper/2306.03974/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 93 references that are in the map.
- First posted on arXiv 6 Jun 2023.

## Builds on (most cited first, 30 of 46)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [GPT Understands, Too](https://mlsanity.com/paper/2103.10385/) (2021)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [A Survey on Deep Learning for Named Entity Recognition](https://mlsanity.com/paper/1812.09449/) (2018)
- [Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification](https://mlsanity.com/paper/2108.02035/) (2021)
- [KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction](https://mlsanity.com/paper/2104.07650/) (2021)
- [Template-Based Named Entity Recognition Using BART](https://mlsanity.com/paper/2106.01760/) (2021)
- [Named Entity Recognition as Dependency Parsing](https://mlsanity.com/paper/2005.07150/) (2020)
- [Unified Named Entity Recognition as Word-Word Relation Classification](https://mlsanity.com/paper/2112.10070/) (2021)
- [Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Network](https://mlsanity.com/paper/2006.05702/) (2020)
- [Template-free Prompt Tuning for Few-shot NER](https://mlsanity.com/paper/2109.13532/) (2021)
- [CONTaiNER: Few-Shot Named Entity Recognition via Contrastive Learning](https://mlsanity.com/paper/2109.07589/) (2021)
- [Cross-Lingual Semantic Role Labeling with High-Quality Translated Training Corpus](https://mlsanity.com/paper/2004.06295/) (2020)
- [LasUIE: Unifying Information Extraction with Latent Adaptive Structure-aware Generative Language Model](https://mlsanity.com/paper/2304.06248/) (2023)
- [Good Examples Make A Faster Learner: Simple Demonstration-based Learning for Low-resource NER](https://mlsanity.com/paper/2110.08454/) (2021)
- [Learn from Syntax: Improving Pair-wise Aspect and Opinion Terms Extractionwith Rich Syntactic Knowledge](https://mlsanity.com/paper/2105.02520/) (2021)
- [LightNER: A Lightweight Tuning Paradigm for Low-resource NER via Pluggable Prompting](https://mlsanity.com/paper/2109.00720/) (2021)
- [Hierarchically-Refined Label Attention Network for Sequence Labeling](https://mlsanity.com/paper/1908.08676/) (2019)
- [Mastering the Explicit Opinion-role Interaction: Syntax-aided Neural Transition System for Unified Opinion Role Labeling](https://mlsanity.com/paper/2110.02001/) (2021)
- [Retrofitting Structure-aware Transformer Language Model for End Tasks](https://mlsanity.com/paper/2009.07408/) (2020)
- [NSP-BERT: A Prompt-based Few-Shot Learner Through an Original Pre-training Task--Next Sentence Prediction](https://mlsanity.com/paper/2109.03564/) (2021)
- [End-to-end Semantic Role Labeling with Neural Transition-based Model](https://mlsanity.com/paper/2101.00394/) (2021)
- [Effective Token Graph Modeling using a Novel Labeling Strategy for Structured Sentiment Analysis](https://mlsanity.com/paper/2203.10796/) (2022)
- [OpenHowNet: An Open Sememe-based Lexical Knowledge Base](https://mlsanity.com/paper/1901.09957/) (2019)

## Built on by (most cited first, 2 of 2)

- [Syntactic Fusion: Enhancing Aspect-Level Sentiment Analysis Through Multi-Tree Graph Integration](https://mlsanity.com/paper/2312.03738/) (2023)
- [Syntax-Informed Interactive Model for Comprehensive Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2312.03739/) (2023)

## Abstract (arXiv)

Few-shot named entity recognition (NER) exploits limited annotated instances to identify named mentions. Effectively transferring the internal or external resources thus becomes the key to few-shot NER. While the existing prompt tuning methods have shown remarkable few-shot performances, they still fail to make full use of knowledge. In this work, we investigate the integration of rich knowledge to prompt tuning for stronger few-shot NER. We propose incorporating the deep prompt tuning framework with threefold knowledge (namely TKDP), including the internal 1) context knowledge and the external 2) label knowledge & 3) sememe knowledge. TKDP encodes the three feature sources and incorporates them into the soft prompt embeddings, which are further injected into an existing pre-trained language model to facilitate predictions. On five benchmark datasets, our knowledge-enriched model boosts by at most 11.53% F1 over the raw deep prompt method, and significantly outperforms 8 strong-performing baseline systems in 5-/10-/20-shot settings, showing great potential in few-shot NER. Our TKDP can be broadly adapted to other few-shot tasks without effort.