# Retrieval Augmented Instruction Tuning for Open NER with Large Language Models

arXiv: https://arxiv.org/abs/2406.17305 · HTML: https://mlsanity.com/paper/2406.17305/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation.
- Builds on 20 of its 34 references that are in the map.
- First posted on arXiv 25 Jun 2024.

## Placed in

- Retrieval-augmented generation / [Generation and context integration](https://mlsanity.com/rag/generation-and-context-integration/)

## Builds on (most cited first, 20 of 20)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023)
- [In-Context Retrieval-Augmented Language Models](https://mlsanity.com/paper/2302.00083/) (2023)
- [RAFT: Adapting Language Model to Domain Specific RAG](https://mlsanity.com/paper/2403.10131/) (2024)
- [RA-DIT: Retrieval-Augmented Dual Instruction Tuning](https://mlsanity.com/paper/2310.01352/) (2023)
- [InstructUIE: Multi-task Instruction Tuning for Unified Information Extraction](https://mlsanity.com/paper/2304.08085/) (2023)
- [GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction](https://mlsanity.com/paper/2310.03668/) (2023)
- [Skywork: A More Open Bilingual Foundation Model](https://mlsanity.com/paper/2310.19341/) (2023)
- [MoDS: Model-oriented Data Selection for Instruction Tuning](https://mlsanity.com/paper/2311.15653/) (2023)
- [CrossNER: Evaluating Cross-Domain Named Entity Recognition](https://mlsanity.com/paper/2012.04373/) (2020)
- [CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark](https://mlsanity.com/paper/2106.08087/) (2021)
- [ChatQA: Surpassing GPT-4 on Conversational QA and RAG](https://mlsanity.com/paper/2401.10225/) (2024)
- [Exploiting Asymmetry for Synthetic Training Data Generation: SynthIE and the Case of Information Extraction](https://mlsanity.com/paper/2303.04132/) (2023)
- [LLMaAA: Making Large Language Models as Active Annotators](https://mlsanity.com/paper/2310.19596/) (2023)
- [Empirical Study of Zero-Shot NER with ChatGPT](https://mlsanity.com/paper/2310.10035/) (2023)
- [Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimation](https://mlsanity.com/paper/2402.18191/) (2024)
- [Retrieval-Augmented Code Generation for Universal Information Extraction](https://mlsanity.com/paper/2311.02962/) (2023)
- [KnowCoder: Coding Structured Knowledge into LLMs for Universal Information Extraction](https://mlsanity.com/paper/2403.07969/) (2024)
- [CLUENER2020: Fine-grained Named Entity Recognition Dataset and Benchmark for Chinese](https://mlsanity.com/paper/2001.04351/) (2020)

## Built on by (most cited first, 4 of 4)

- [Instruction Finetuning DeepSeek-R1-8B Model Using LoRA and NEFTune](https://mlsanity.com/paper/2606.10392/) (2026)
- [ToMMeR -- Efficient Entity Mention Detection from Large Language Models](https://mlsanity.com/paper/2510.19410/) (2025)
- [Instruction Finetuning LLaMA-3-8B Model Using LoRA for Financial Named Entity Recognition](https://mlsanity.com/paper/2601.10043/) (2026)
- [Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoning](https://mlsanity.com/paper/2610.01936/) (2026)

## Abstract (arXiv)

The strong capability of large language models (LLMs) has been applied to information extraction (IE) through either retrieval augmented prompting or instruction tuning (IT). However, the best way to incorporate information with LLMs for IE remains an open question. In this paper, we explore Retrieval Augmented Instruction Tuning (RA-IT) for IE, focusing on the task of open named entity recognition (NER). Specifically, for each training sample, we retrieve semantically similar examples from the training dataset as the context and prepend them to the input of the original instruction. To evaluate our RA-IT approach more thoroughly, we construct a Chinese IT dataset for open NER and evaluate RA-IT in both English and Chinese scenarios. Experimental results verify the effectiveness of RA-IT across various data sizes and in both English and Chinese scenarios. We also conduct thorough studies to explore the impacts of various retrieval strategies in the proposed RA-IT framework. Code and data are available at: https://github.com/Emma1066/Retrieval-Augmented-IT-OpenNER