# LinkNER: Linking Local Named Entity Recognition Models to Large Language Models using Uncertainty

arXiv: https://arxiv.org/abs/2402.10573 · HTML: https://mlsanity.com/paper/2402.10573/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 40 references that are in the map.
- First posted on arXiv 16 Feb 2024.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Evidential Deep Learning to Quantify Classification Uncertainty](https://mlsanity.com/paper/1806.01768/) (2018)
- [Unified Named Entity Recognition as Word-Word Relation Classification](https://mlsanity.com/paper/2112.10070/) (2021)
- [Chinese NER Using Lattice LSTM](https://mlsanity.com/paper/1805.02023/) (2018)
- [An Analysis of Simple Data Augmentation for Named Entity Recognition](https://mlsanity.com/paper/2010.11683/) (2020)
- [SpanNER: Named Entity Re-/Recognition as Span Prediction](https://mlsanity.com/paper/2106.00641/) (2021)
- [Uncertainty in Natural Language Processing: Sources, Quantification, and Applications](https://mlsanity.com/paper/2306.04459/) (2023)
- [Small Models are Valuable Plug-ins for Large Language Models](https://mlsanity.com/paper/2305.08848/) (2023)
- [Rethinking Generalization of Neural Models: A Named Entity Recognition Case Study](https://mlsanity.com/paper/2001.03844/) (2020)
- [A Trigger-Sense Memory Flow Framework for Joint Entity and Relation Extraction](https://mlsanity.com/paper/2101.10213/) (2021)
- [MINER: Improving Out-of-Vocabulary Named Entity Recognition from an Information Theoretic Perspective](https://mlsanity.com/paper/2204.04391/) (2022)
- [Interpretability Analysis for Named Entity Recognition to Understand System Predictions and How They Can Improve](https://mlsanity.com/paper/2004.04564/) (2020)

## Built on by (most cited first, 12 of 12)

- [A Survey of Uncertainty Estimation in LLMs: Theory Meets Practice](https://mlsanity.com/paper/2410.15326/) (2024)
- [Order Matters: Investigate the Position Bias in Multi-constraint Instruction Following](https://mlsanity.com/paper/2502.17204/) (2025)
- [A Cooperative Multi-Agent Framework for Zero-Shot Named Entity Recognition](https://mlsanity.com/paper/2502.18702/) (2025)
- [Explainable LLM-driven Multi-dimensional Distillation for E-Commerce Relevance Learning](https://mlsanity.com/paper/2411.13045/) (2024)
- [A Benchmark and Robustness Study of In-Context-Learning with Large Language Models in Music Entity Detection](https://mlsanity.com/paper/2412.11851/) (2024)
- [Can Large Language Models Really Recognize Your Name?](https://mlsanity.com/paper/2505.14549/) (2025)
- [BANER: Boundary-Aware LLMs for Few-Shot Named Entity Recognition](https://mlsanity.com/paper/2412.02228/) (2024)
- [ReFineG: Synergizing Small Supervised Models and LLMs for Low-Resource Grounded Multimodal NER](https://mlsanity.com/paper/2509.10975/) (2025)
- [Do LLMs Judge Distantly Supervised Named Entity Labels Well? Constructing the JudgeWEL Dataset](https://mlsanity.com/paper/2601.00411/) (2026)
- [Software Mention Recognition with a Three-Stage Framework Based on BERTology Models at SOMD 2024](https://mlsanity.com/paper/2405.01575/) (2024)
- [Uncertainty Quantification for Named Entity Recognition via Full-Sequence and Subsequence Conformal Prediction](https://mlsanity.com/paper/2601.16999/) (2026)
- [Enhancing Generative Information Extraction with Two-step Validation: A Product Attribute Use Case](https://mlsanity.com/paper/2607.26780/) (2026)

## Abstract (arXiv)

Named Entity Recognition (NER) serves as a fundamental task in natural language understanding, bearing direct implications for web content analysis, search engines, and information retrieval systems. Fine-tuned NER models exhibit satisfactory performance on standard NER benchmarks. However, due to limited fine-tuning data and lack of knowledge, it performs poorly on unseen entity recognition. As a result, the usability and reliability of NER models in web-related applications are compromised. Instead, Large Language Models (LLMs) like GPT-4 possess extensive external knowledge, but research indicates that they lack specialty for NER tasks. Furthermore, non-public and large-scale weights make tuning LLMs difficult. To address these challenges, we propose a framework that combines small fine-tuned models with LLMs (LinkNER) and an uncertainty-based linking strategy called RDC that enables fine-tuned models to complement black-box LLMs, achieving better performance. We experiment with both standard NER test sets and noisy social media datasets. LinkNER enhances NER task performance, notably surpassing SOTA models in robustness tests. We also quantitatively analyze the influence of key components like uncertainty estimation methods, LLMs, and in-context learning on diverse NER tasks, offering specific web-related recommendations. Code is available at https://github.com/zhzhengit/LinkNER.