# LLM-IE: A Python Package for Generative Information Extraction with Large Language Models

arXiv: https://arxiv.org/abs/2411.11779 · HTML: https://mlsanity.com/paper/2411.11779/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 25 references that are in the map.
- First posted on arXiv 18 Nov 2024.

## Builds on (most cited first, 10 of 10)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Large Language Models are Few-Shot Clinical Information Extractors](https://mlsanity.com/paper/2205.12689/) (2022)
- [Large Language Models for Generative Information Extraction: A Survey](https://mlsanity.com/paper/2312.17617/) (2023)
- [InstructUIE: Multi-task Instruction Tuning for Unified Information Extraction](https://mlsanity.com/paper/2304.08085/) (2023)
- [Revisiting Relation Extraction in the era of Large Language Models](https://mlsanity.com/paper/2305.05003/) (2023)
- [Evaluation of ChatGPT on Biomedical Tasks: A Zero-Shot Comparison with Fine-Tuned Generative Transformers](https://mlsanity.com/paper/2306.04504/) (2023)
- [Self-Improving for Zero-Shot Named Entity Recognition with Large Language Models](https://mlsanity.com/paper/2311.08921/) (2023)
- [Leveraging Large Language Models for Knowledge-free Weak Supervision in Clinical Natural Language Processing](https://mlsanity.com/paper/2406.06723/) (2024)

## Built on by (most cited first, 4 of 4)

- [A Survey on Large Language Models in Multimodal Recommender Systems](https://mlsanity.com/paper/2505.09777/) (2025)
- [CienaLLM: Generative Climate-Impact Extraction from News Articles with Autoregressive LLMs](https://mlsanity.com/paper/2512.19305/) (2025)
- [A Multi-Agent System for Autonomous, Fine-Tuning-Free Clinical Symptom Detection: Development and Validation Study](https://mlsanity.com/paper/2607.12886/) (2026)
- [Toward a Locally Deployable Agentic Co-Scientist: Small-Model Planning for Early-Stage Drug Discovery](https://mlsanity.com/paper/2610.04740/) (2026)

## Abstract (arXiv)

Objectives: Despite the recent adoption of large language models (LLMs) for biomedical information extraction, challenges in prompt engineering and algorithms persist, with no dedicated software available. To address this, we developed LLM-IE: a Python package for building complete information extraction pipelines. Our key innovation is an interactive LLM agent to support schema definition and prompt design. Materials and Methods: The LLM-IE supports named entity recognition, entity attribute extraction, and relation extraction tasks. We benchmarked on the i2b2 datasets and conducted a system evaluation. Results: The sentence-based prompting algorithm resulted in the best performance while requiring a longer inference time. System evaluation provided intuitive visualization. Discussion: LLM-IE was designed from practical NLP experience in healthcare and has been adopted in internal projects. It should hold great value to the biomedical NLP community. Conclusion: We developed a Python package, LLM-IE, that provides building blocks for robust information extraction pipeline construction.