# IEPile: Unearthing Large-Scale Schema-Based Information Extraction Corpus

arXiv: https://arxiv.org/abs/2402.14710 · HTML: https://mlsanity.com/paper/2402.14710/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 96 references that are in the map.
- First posted on arXiv 22 Feb 2024.

## Builds on (most cited first, 30 of 53)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://mlsanity.com/paper/2103.10360/) (2021)
- [Deduplicating Training Data Makes Language Models Better](https://mlsanity.com/paper/2107.06499/) (2021)
- [Baichuan 2: Open Large-scale Language Models](https://mlsanity.com/paper/2309.10305/) (2023)
- [Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction](https://mlsanity.com/paper/1808.09602/) (2018)
- [FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation](https://mlsanity.com/paper/1810.10147/) (2018)
- [ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPT](https://mlsanity.com/paper/2302.10205/) (2023)
- [Unified Structure Generation for Universal Information Extraction](https://mlsanity.com/paper/2203.12277/) (2022)
- [A Unified MRC Framework for Named Entity Recognition](https://mlsanity.com/paper/1910.11476/) (2019)
- [Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2311.13165/) (2023)
- [Large Language Models for Generative Information Extraction: A Survey](https://mlsanity.com/paper/2312.17617/) (2023)
- [Structured Prediction as Translation between Augmented Natural Languages](https://mlsanity.com/paper/2101.05779/) (2021)
- [KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction](https://mlsanity.com/paper/2104.07650/) (2021)
- [InstructUIE: Multi-task Instruction Tuning for Unified Information Extraction](https://mlsanity.com/paper/2304.08085/) (2023)
- [Document-Level Event Argument Extraction by Conditional Generation](https://mlsanity.com/paper/2104.05919/) (2021)
- [Prompting PaLM for Translation: Assessing Strategies and Performance](https://mlsanity.com/paper/2211.09102/) (2022)
- [Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness](https://mlsanity.com/paper/2304.11633/) (2023)
- [Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples!](https://mlsanity.com/paper/2303.08559/) (2023)
- [GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction](https://mlsanity.com/paper/2310.03668/) (2023)
- [CrossNER: Evaluating Cross-Domain Named Entity Recognition](https://mlsanity.com/paper/2012.04373/) (2020)
- [Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme](https://mlsanity.com/paper/1706.05075/) (2017)
- [Revisiting Relation Extraction in the era of Large Language Models](https://mlsanity.com/paper/2305.05003/) (2023)
- [A Span-Extraction Dataset for Chinese Machine Reading Comprehension](https://mlsanity.com/paper/1810.07366/) (2018)
- [Chinese NER Using Lattice LSTM](https://mlsanity.com/paper/1805.02023/) (2018)
- [Multi-Sentence Argument Linking](https://mlsanity.com/paper/1911.03766/) (2019)

## Built on by (most cited first, 5 of 5)

- [KAG: Boosting LLMs in Professional Domains via Knowledge Augmented Generation](https://mlsanity.com/paper/2409.13731/) (2024)
- [Beyond Boundaries: Learning a Universal Entity Taxonomy across Datasets and Languages for Open Named Entity Recognition](https://mlsanity.com/paper/2406.11192/) (2024)
- [Improving Natural Language Understanding for LLMs via Large-Scale Instruction Synthesis](https://mlsanity.com/paper/2502.03843/) (2025)
- [KnowCoder-V2: Deep Knowledge Analysis](https://mlsanity.com/paper/2506.06881/) (2025)
- [DeepJSONEval: Benchmarking Complex Nested JSON Data Mining for Large Language Models](https://mlsanity.com/paper/2509.25922/) (2025)

## Abstract (arXiv)

Large Language Models (LLMs) demonstrate remarkable potential across various domains; however, they exhibit a significant performance gap in Information Extraction (IE). Note that high-quality instruction data is the vital key for enhancing the specific capabilities of LLMs, while current IE datasets tend to be small in scale, fragmented, and lack standardized schema. To this end, we introduce IEPile, a comprehensive bilingual (English and Chinese) IE instruction corpus, which contains approximately 0.32B tokens. We construct IEPile by collecting and cleaning 33 existing IE datasets, and introduce schema-based instruction generation to unearth a large-scale corpus. Experimentally, IEPile enhance the performance of LLMs for IE, with notable improvements in zero-shot generalization. We open-source the resource and pre-trained models, hoping to provide valuable support to the NLP community.