# InstructUIE: Multi-task Instruction Tuning for Unified Information Extraction

arXiv: https://arxiv.org/abs/2304.08085 · HTML: https://mlsanity.com/paper/2304.08085/

## Key facts

- Cited by 170 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 71 references that are in the map.
- First posted on arXiv 17 Apr 2023.

## Builds on (most cited first, 30 of 39)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks](https://mlsanity.com/paper/2204.07705/) (2022)
- [The Flan Collection: Designing Data and Methods for Effective Instruction Tuning](https://mlsanity.com/paper/2301.13688/) (2023)
- [Cross-Task Generalization via Natural Language Crowdsourcing Instructions](https://mlsanity.com/paper/2104.08773/) (2021)
- [Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction](https://mlsanity.com/paper/1808.09602/) (2018)
- [FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation](https://mlsanity.com/paper/1810.10147/) (2018)
- [A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Models](https://mlsanity.com/paper/2303.10420/) (2023)
- [Unified Structure Generation for Universal Information Extraction](https://mlsanity.com/paper/2203.12277/) (2022)
- [Dice Loss for Data-imbalanced NLP Tasks](https://mlsanity.com/paper/1911.02855/) (2019)
- [Text2Event: Controllable Sequence-to-Structure Generation for End-to-end Event Extraction](https://mlsanity.com/paper/2106.09232/) (2021)
- [CrossNER: Evaluating Cross-Domain Named Entity Recognition](https://mlsanity.com/paper/2012.04373/) (2020)
- [A Unified Generative Framework for Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2106.04300/) (2021)
- [Automated Concatenation of Embeddings for Structured Prediction](https://mlsanity.com/paper/2010.05006/) (2020)
- [RelationPrompt: Leveraging Prompts to Generate Synthetic Data for Zero-Shot Relation Triplet Extraction](https://mlsanity.com/paper/2203.09101/) (2022)
- [How Robust is GPT-3.5 to Predecessors? A Comprehensive Study on Language Understanding Tasks](https://mlsanity.com/paper/2303.00293/) (2023)
- [A Hierarchical Framework for Relation Extraction with Reinforcement Learning](https://mlsanity.com/paper/1811.03925/) (2018)
- [Universal Information Extraction as Unified Semantic Matching](https://mlsanity.com/paper/2301.03282/) (2023)
- [Relation Classification via Recurrent Neural Network](https://mlsanity.com/paper/1508.01006/) (2015)
- [ZS-BERT: Towards Zero-Shot Relation Extraction with Attribute Representation Learning](https://mlsanity.com/paper/2104.04697/) (2021)
- [FLERT: Document-Level Features for Named Entity Recognition](https://mlsanity.com/paper/2011.06993/) (2020)
- [DeepStruct: Pretraining of Language Models for Structure Prediction](https://mlsanity.com/paper/2205.10475/) (2022)
- [UniRE: A Unified Label Space for Entity Relation Extraction](https://mlsanity.com/paper/2107.04292/) (2021)
- [A Frustratingly Easy Approach for Entity and Relation Extraction](https://mlsanity.com/paper/2010.12812/) (2020)
- [T-NER: An All-Round Python Library for Transformer-based Named Entity Recognition](https://mlsanity.com/paper/2209.12616/) (2022)
- [PHEE: A Dataset for Pharmacovigilance Event Extraction from Text](https://mlsanity.com/paper/2210.12560/) (2022)
- [Improving Distantly Supervised Relation Extraction using Word and Entity Based Attention](https://mlsanity.com/paper/1804.06987/) (2018)
- [POLYGLOT-NER: Massive Multilingual Named Entity Recognition](https://mlsanity.com/paper/1410.3791/) (2014)

## Built on by (most cited first, 30 of 170)

- [Instruction Tuning for Large Language Models: A Survey](https://mlsanity.com/paper/2308.10792/) (2023)
- [Orthogonal Subspace Learning for Language Model Continual Learning](https://mlsanity.com/paper/2310.14152/) (2023)
- [FireAct: Toward Language Agent Fine-tuning](https://mlsanity.com/paper/2310.05915/) (2023)
- [Large Language Models for Generative Information Extraction: A Survey](https://mlsanity.com/paper/2312.17617/) (2023)
- [Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness](https://mlsanity.com/paper/2304.11633/) (2023)
- [GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction](https://mlsanity.com/paper/2310.03668/) (2023)
- [Large Language Models Meet NLP: A Survey](https://mlsanity.com/paper/2405.12819/) (2024)
- [Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Models](https://mlsanity.com/paper/2403.12881/) (2024)
- [GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformer](https://mlsanity.com/paper/2311.08526/) (2023)
- [UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition](https://mlsanity.com/paper/2308.03279/) (2023)
- [Large Language Models and Knowledge Graphs: Opportunities and Challenges](https://mlsanity.com/paper/2308.06374/) (2023)
- [InstructGraph: Boosting Large Language Models via Graph-centric Instruction Tuning and Preference Alignment](https://mlsanity.com/paper/2402.08785/) (2024)
- [An Empirical Study on Information Extraction using Large Language Models](https://mlsanity.com/paper/2305.14450/) (2023)
- [PiVe: Prompting with Iterative Verification Improving Graph-based Generative Capability of LLMs](https://mlsanity.com/paper/2305.12392/) (2023)
- [ChatKBQA: A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Models](https://mlsanity.com/paper/2310.08975/) (2023)
- [SciRIFF: A Resource to Enhance Language Model Instruction-Following over Scientific Literature](https://mlsanity.com/paper/2406.07835/) (2024)
- [Retrieval-Augmented Code Generation for Universal Information Extraction](https://mlsanity.com/paper/2311.02962/) (2023)
- [Unveiling the Generalization Power of Fine-Tuned Large Language Models](https://mlsanity.com/paper/2403.09162/) (2024)
- [NuNER: Entity Recognition Encoder Pre-training via LLM-Annotated Data](https://mlsanity.com/paper/2402.15343/) (2024)
- [KnowCoder: Coding Structured Knowledge into LLMs for Universal Information Extraction](https://mlsanity.com/paper/2403.07969/) (2024)
- [Multi-Task Instruction Tuning of LLaMa for Specific Scenarios: A Preliminary Study on Writing Assistance](https://mlsanity.com/paper/2305.13225/) (2023)
- [Text-Tuple-Table: Towards Information Integration in Text-to-Table Generation via Global Tuple Extraction](https://mlsanity.com/paper/2404.14215/) (2024)
- [Revisiting Large Language Models as Zero-shot Relation Extractors](https://mlsanity.com/paper/2310.05028/) (2023)
- [TextEE: Benchmark, Reevaluation, Reflections, and Future Challenges in Event Extraction](https://mlsanity.com/paper/2311.09562/) (2023)
- [Looking Right is Sometimes Right: Investigating the Capabilities of Decoder-only LLMs for Sequence Labeling](https://mlsanity.com/paper/2401.14556/) (2024)
- [ADELIE: Aligning Large Language Models on Information Extraction](https://mlsanity.com/paper/2405.05008/) (2024)
- [GrammarGPT: Exploring Open-Source LLMs for Native Chinese Grammatical Error Correction with Supervised Fine-Tuning](https://mlsanity.com/paper/2307.13923/) (2023)
- [C-ICL: Contrastive In-context Learning for Information Extraction](https://mlsanity.com/paper/2402.11254/) (2024)
- [Label Supervised LLaMA Finetuning](https://mlsanity.com/paper/2310.01208/) (2023)
- [YAYI-UIE: A Chat-Enhanced Instruction Tuning Framework for Universal Information Extraction](https://mlsanity.com/paper/2312.15548/) (2023)

## Abstract (arXiv)

Large language models have unlocked strong multi-task capabilities from reading instructive prompts. However, recent studies have shown that existing large models still have difficulty with information extraction tasks. For example, gpt-3.5-turbo achieved an F1 score of 18.22 on the Ontonotes dataset, which is significantly lower than the state-of-the-art performance. In this paper, we propose InstructUIE, a unified information extraction framework based on instruction tuning, which can uniformly model various information extraction tasks and capture the inter-task dependency. To validate the proposed method, we introduce IE INSTRUCTIONS, a benchmark of 32 diverse information extraction datasets in a unified text-to-text format with expert-written instructions. Experimental results demonstrate that our method achieves comparable performance to Bert in supervised settings and significantly outperforms the state-of-the-art and gpt3.5 in zero-shot settings.