# Self-Improving for Zero-Shot Named Entity Recognition with Large Language Models

arXiv: https://arxiv.org/abs/2311.08921 · HTML: https://mlsanity.com/paper/2311.08921/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Self-training for few-shot NLP, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 17 of its 27 references that are in the map.
- First posted on arXiv 15 Nov 2023.

## Placed in

- Synthetic data and self-improvement for LLMs / [Self-training for few-shot NLP](https://mlsanity.com/synth/self-training-for-few-shot-nlp/)

## Builds on (most cited first, 17 of 17)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023)
- [Larger language models do in-context learning differently](https://mlsanity.com/paper/2303.03846/) (2023)
- [ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPT](https://mlsanity.com/paper/2302.10205/) (2023)
- [Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness](https://mlsanity.com/paper/2304.11633/) (2023)
- [GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction](https://mlsanity.com/paper/2310.03668/) (2023)
- [A General Framework for Information Extraction using Dynamic Span Graphs](https://mlsanity.com/paper/1904.03296/) (2019)
- [Exploiting Asymmetry for Synthetic Training Data Generation: SynthIE and the Case of Information Extraction](https://mlsanity.com/paper/2303.04132/) (2023)
- [LLMaAA: Making Large Language Models as Active Annotators](https://mlsanity.com/paper/2310.19596/) (2023)
- [Z-ICL: Zero-Shot In-Context Learning with Pseudo-Demonstrations](https://mlsanity.com/paper/2212.09865/) (2022)
- [CodeIE: Large Code Generation Models are Better Few-Shot Information Extractors](https://mlsanity.com/paper/2305.05711/) (2023)
- [Empirical Study of Zero-Shot NER with ChatGPT](https://mlsanity.com/paper/2310.10035/) (2023)
- [An Empirical Study on Information Extraction using Large Language Models](https://mlsanity.com/paper/2305.14450/) (2023)

## Built on by (most cited first, 15 of 15)

- [Large Language Models for Generative Information Extraction: A Survey](https://mlsanity.com/paper/2312.17617/) (2023)
- [LTNER: Large Language Model Tagging for Named Entity Recognition with Contextualized Entity Marking](https://mlsanity.com/paper/2404.05624/) (2024)
- [A Cooperative Multi-Agent Framework for Zero-Shot Named Entity Recognition](https://mlsanity.com/paper/2502.18702/) (2025)
- [Evaluating Zero-Shot Multilingual Aspect-Based Sentiment Analysis with Large Language Models](https://mlsanity.com/paper/2412.12564/) (2024)
- [LLM-IE: A Python Package for Generative Information Extraction with Large Language Models](https://mlsanity.com/paper/2411.11779/) (2024)
- [WhisperNER: Unified Open Named Entity and Speech Recognition](https://mlsanity.com/paper/2409.08107/) (2024)
- [EL4NER: Ensemble Learning for Named Entity Recognition via Multiple Small-Parameter Large Language Models](https://mlsanity.com/paper/2505.23038/) (2025)
- [ReverseNER: A Self-Generated Example-Driven Framework for Zero-Shot Named Entity Recognition with Large Language Models](https://mlsanity.com/paper/2411.00533/) (2024)
- [Few Dimensions are Enough: Fine-tuning BERT with Selected Dimensions Revealed Its Redundant Nature](https://mlsanity.com/paper/2504.04966/) (2025)
- [Automated Constraint Specification for Job Scheduling by Regulating Generative Model with Domain-Specific Representation](https://mlsanity.com/paper/2510.02679/) (2025)
- [OEMA: Ontology-Enhanced Multi-Agent Collaboration Framework for Zero-Shot Clinical Named Entity Recognition](https://mlsanity.com/paper/2511.15211/) (2025)
- [Enhancing Software-Related Information Extraction via Single-Choice Question Answering with Large Language Models](https://mlsanity.com/paper/2404.05587/) (2024)
- [Towards Refining Developer Questions using LLM-Based Named Entity Recognition for Developer Chatroom Conversations](https://mlsanity.com/paper/2503.00673/) (2025)
- [CodeNER: Code Prompting for Named Entity Recognition](https://mlsanity.com/paper/2507.20423/) (2025)
- [Named Entity Recognition of Historical Texts via Large Language Model](https://mlsanity.com/paper/2508.18090/) (2025)

## Abstract (arXiv)

Exploring the application of powerful large language models (LLMs) on the named entity recognition (NER) task has drawn much attention recently. This work pushes the performance boundary of zero-shot NER with LLMs by proposing a training-free self-improving framework, which utilizes an unlabeled corpus to stimulate the self-learning ability of LLMs. First, we use the LLM to make predictions on the unlabeled corpus using self-consistency and obtain a self-annotated dataset. Second, we explore various strategies to select reliable annotations to form a reliable self-annotated dataset. Finally, for each test input, we retrieve demonstrations from the reliable self-annotated dataset and perform inference via in-context learning. Experiments on four benchmarks show substantial performance improvements achieved by our framework. Through comprehensive experimental analysis, we find that increasing the size of unlabeled corpus or iterations of self-improving does not guarantee further improvement, but the performance might be boosted via more advanced strategies for reliable annotation selection. Code and data are publicly available at https://github.com/Emma1066/Self-Improve-Zero-Shot-NER