# InstructionNER: A Multi-Task Instruction-Based Generative Framework for Few-shot NER

arXiv: https://arxiv.org/abs/2203.03903 · HTML: https://mlsanity.com/paper/2203.03903/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 34 references that are in the map.
- First posted on arXiv 8 Mar 2022.

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://mlsanity.com/paper/2001.07676/) (2020)
- [Cross-Task Generalization via Natural Language Crowdsourcing Instructions](https://mlsanity.com/paper/2104.08773/) (2021)
- [End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF](https://mlsanity.com/paper/1603.01354/) (2016)
- [Pre-Trained Models: Past, Present and Future](https://mlsanity.com/paper/2106.07139/) (2021)
- [Named Entity Recognition with Bidirectional LSTM-CNNs](https://mlsanity.com/paper/1511.08308/) (2015)
- [Neural Approaches to Conversational AI](https://mlsanity.com/paper/1809.08267/) (2018)
- [Template-Based Named Entity Recognition Using BART](https://mlsanity.com/paper/2106.01760/) (2021)
- [A Unified Generative Framework for Various NER Subtasks](https://mlsanity.com/paper/2106.01223/) (2021)
- [Neural Architectures for Nested NER through Linearization](https://mlsanity.com/paper/1908.06926/) (2019)
- [Few-Shot Named Entity Recognition: A Comprehensive Study](https://mlsanity.com/paper/2012.14978/) (2020)
- [Fast and Accurate Entity Recognition with Iterated Dilated Convolutions](https://mlsanity.com/paper/1702.02098/) (2017)
- [An Effective Transition-based Model for Discontinuous NER](https://mlsanity.com/paper/2004.13454/) (2020)
- [Noisy-Labeled NER with Confidence Estimation](https://mlsanity.com/paper/2104.04318/) (2021)
- [Label-Agnostic Sequence Labeling by Copying Nearest Neighbors](https://mlsanity.com/paper/1906.04225/) (2019)
- [GCDT: A Global Context Enhanced Deep Transition Architecture for Sequence Labeling](https://mlsanity.com/paper/1906.02437/) (2019)
- [Example-Based Named Entity Recognition](https://mlsanity.com/paper/2008.10570/) (2020)

## Built on by (most cited first, 30 of 31)

- [Large Language Models for Generative Information Extraction: A Survey](https://mlsanity.com/paper/2312.17617/) (2023)
- [InstructUIE: Multi-task Instruction Tuning for Unified Information Extraction](https://mlsanity.com/paper/2304.08085/) (2023)
- [LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models](https://mlsanity.com/paper/2404.15522/) (2024)
- [Large Language Model Instruction Following: A Survey of Progresses and Challenges](https://mlsanity.com/paper/2303.10475/) (2023)
- [Looking Right is Sometimes Right: Investigating the Capabilities of Decoder-only LLMs for Sequence Labeling](https://mlsanity.com/paper/2401.14556/) (2024)
- [InstructABSA: Instruction Learning for Aspect Based Sentiment Analysis](https://mlsanity.com/paper/2302.08624/) (2023)
- [Instruction Tuned Models are Quick Learners](https://mlsanity.com/paper/2306.05539/) (2023)
- [HELP ME THINK: A Simple Prompting Strategy for Non-experts to Create Customized Content with Models](https://mlsanity.com/paper/2208.08232/) (2022)
- [How Many Data Samples is an Additional Instruction Worth?](https://mlsanity.com/paper/2203.09161/) (2022)
- [In-BoXBART: Get Instructions into Biomedical Multi-Task Learning](https://mlsanity.com/paper/2204.07600/) (2022)
- [InstructExcel: A Benchmark for Natural Language Instruction in Excel](https://mlsanity.com/paper/2310.14495/) (2023)
- [BioTABQA: Instruction Learning for Biomedical Table Question Answering](https://mlsanity.com/paper/2207.02419/) (2022)
- [NERetrieve: Dataset for Next Generation Named Entity Recognition and Retrieval](https://mlsanity.com/paper/2310.14282/) (2023)
- [PSSAT: A Perturbed Semantic Structure Awareness Transferring Method for Perturbation-Robust Slot Filling](https://mlsanity.com/paper/2208.11508/) (2022)
- [Revisit Out-Of-Vocabulary Problem for Slot Filling: A Unified Contrastive Frameword with Multi-level Data Augmentations](https://mlsanity.com/paper/2302.13584/) (2023)
- [GIELLM: Japanese General Information Extraction Large Language Model Utilizing Mutual Reinforcement Effect](https://mlsanity.com/paper/2311.06838/) (2023)
- [DemoNSF: A Multi-task Demonstration-based Generative Framework for Noisy Slot Filling Task](https://mlsanity.com/paper/2310.10169/) (2023)
- [A Unified Framework for Multi-intent Spoken Language Understanding with prompting](https://mlsanity.com/paper/2210.03337/) (2022)
- [Exploring the Limits of Transfer Learning with Unified Model in the Cybersecurity Domain](https://mlsanity.com/paper/2302.10346/) (2023)
- [Sentence-to-Label Generation Framework for Multi-task Learning of Japanese Sentence Classification and Named Entity Recognition](https://mlsanity.com/paper/2306.15978/) (2023)
- [Evaluating Named Entity Recognition: A comparative analysis of mono- and multilingual transformer models on a novel Brazilian corporate earnings call transcripts dataset](https://mlsanity.com/paper/2403.12212/) (2024)
- [Challenges and Opportunities of Using Transformer-Based Multi-Task Learning in NLP Through ML Lifecycle: A Survey](https://mlsanity.com/paper/2308.08234/) (2023)
- [Optimal strategies to perform multilingual analysis of social content for a novel dataset in the tourism domain](https://mlsanity.com/paper/2311.14727/) (2023)
- [Noise-BERT: A Unified Perturbation-Robust Framework with Noise Alignment Pre-training for Noisy Slot Filling Task](https://mlsanity.com/paper/2402.14494/) (2024)
- [Supervised In-Context Fine-Tuning for Generative Sequence Labeling](https://mlsanity.com/paper/2509.00921/) (2025)
- [A Study of Large Language Models for Patient Information Extraction: Model Architecture, Fine-Tuning Strategy, and Multi-task Instruction Tuning](https://mlsanity.com/paper/2509.04753/) (2025)
- [Large Language Models for Geolocation Extraction in Humanitarian Crisis Response](https://mlsanity.com/paper/2602.08872/) (2026)
- [GLiNER-Relex: A Unified Framework for Joint Named Entity Recognition and Relation Extraction](https://mlsanity.com/paper/2605.10108/) (2026)
- [Mutual Reinforcement Effects in Japanese Sentence Classification and Named Entity Recognition Tasks](https://mlsanity.com/paper/2307.10291/) (2023)
- [Multi-BERT: Leveraging Adapters and Prompt Tuning for Low-Resource Multi-Domain Adaptation](https://mlsanity.com/paper/2404.02335/) (2024)

## Abstract (arXiv)

Recently, prompt-based methods have achieved significant performance in few-shot learning scenarios by bridging the gap between language model pre-training and fine-tuning for downstream tasks. However, existing prompt templates are mostly designed for sentence-level tasks and are inappropriate for sequence labeling objectives. To address the above issue, we propose a multi-task instruction-based generative framework, named InstructionNER, for low-resource named entity recognition. Specifically, we reformulate the NER task as a generation problem, which enriches source sentences with task-specific instructions and answer options, then inferences the entities and types in natural language. We further propose two auxiliary tasks, including entity extraction and entity typing, which enable the model to capture more boundary information of entities and deepen the understanding of entity type semantics, respectively. Experimental results show that our method consistently outperforms other baselines on five datasets in few-shot settings.