# I Don't Know: Explicit Modeling of Uncertainty with an [IDK] Token

arXiv: https://arxiv.org/abs/2412.06676 · HTML: https://mlsanity.com/paper/2412.06676/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 57 of its 66 references that are in the map.
- First posted on arXiv 9 Dec 2024.

## Builds on (most cited first, 30 of 57)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [On Faithfulness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2005.00661/) (2020)
- [Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation](https://mlsanity.com/paper/2302.09664/) (2023)
- [Teaching Models to Express Their Uncertainty in Words](https://mlsanity.com/paper/2205.14334/) (2022)
- [How Much Knowledge Can You Pack Into the Parameters of a Language Model?](https://mlsanity.com/paper/2002.08910/) (2020)
- [The Internal State of an LLM Knows When It's Lying](https://mlsanity.com/paper/2304.13734/) (2023)
- [How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering](https://mlsanity.com/paper/2012.00955/) (2020)
- [Asking and Answering Questions to Evaluate the Factual Consistency of Summaries](https://mlsanity.com/paper/2004.04228/) (2020)
- [Measuring and Improving Consistency in Pretrained Language Models](https://mlsanity.com/paper/2102.01017/) (2021)
- [Calibration of Pre-trained Transformers](https://mlsanity.com/paper/2003.07892/) (2020)
- [RARR: Researching and Revising What Language Models Say, Using Language Models](https://mlsanity.com/paper/2210.08726/) (2022)
- [Measuring Attribution in Natural Language Generation Models](https://mlsanity.com/paper/2112.12870/) (2021)
- [A Survey of Confidence Estimation and Calibration in Large Language Models](https://mlsanity.com/paper/2311.08298/) (2023)
- [Transformer Memory as a Differentiable Search Index](https://mlsanity.com/paper/2202.06991/) (2022)
- [QuestEval: Summarization Asks for Fact-based Evaluation](https://mlsanity.com/paper/2103.12693/) (2021)
- [Evaluating the Ripple Effects of Knowledge Editing in Language Models](https://mlsanity.com/paper/2307.12976/) (2023)

## Built on by (most cited first, 26 of 26)

- [Generate, but Verify: Reducing Hallucination in Vision-Language Models with Retrospective Resampling](https://mlsanity.com/paper/2504.13169/) (2025)
- [CER: Confidence Enhanced Reasoning in LLMs](https://mlsanity.com/paper/2502.14634/) (2025)
- [AgentAbstain: Do LLM Agents Know When Not to Act?](https://mlsanity.com/paper/2607.10059/) (2026)
- [Query-Level Uncertainty in Large Language Models](https://mlsanity.com/paper/2506.09669/) (2025)
- [When to Speak, When to Abstain: Contrastive Decoding with Abstention](https://mlsanity.com/paper/2412.12527/) (2024)
- [On Calibration of Large Language Models: From Response To Capability](https://mlsanity.com/paper/2602.13540/) (2026)
- [Uncertainty-o: One Model-agnostic Framework for Unveiling Uncertainty in Large Multimodal Models](https://mlsanity.com/paper/2506.07575/) (2025)
- [ConfRAG: Confidence-Guided Retrieval-Augmenting Generation](https://mlsanity.com/paper/2506.07309/) (2025)
- [ReFACT: A Benchmark for Scientific Confabulation Detection with Positional Error Annotations](https://mlsanity.com/paper/2509.25868/) (2025)
- [LLM Performance Predictors: Learning When to Escalate in Hybrid Human-AI Moderation Systems](https://mlsanity.com/paper/2601.07006/) (2026)
- [Steering the Verifiability of Multimodal AI Hallucinations](https://mlsanity.com/paper/2604.06714/) (2026)
- [InFact: Informativeness Alignment for Improved LLM Factuality](https://mlsanity.com/paper/2505.20487/) (2025)
- [SEAT: Sparse Entity-Aware Tuning for Knowledge Adaptation while Preserving Epistemic Abstention](https://mlsanity.com/paper/2506.14387/) (2025)
- [Improving Semantic Uncertainty Quantification in LVLMs with Semantic Gaussian Processes](https://mlsanity.com/paper/2512.14177/) (2025)
- [TIAR: Trajectory-Informed Advantage Reweighting for LLM Abstention Learning](https://mlsanity.com/paper/2605.25850/) (2026)
- [Catch Your Breath: Adaptive Computation for Self-Paced Sequence Production](https://mlsanity.com/paper/2510.13879/) (2025)
- [Neurosymbolic Learning for Inference-Time Argumentation](https://mlsanity.com/paper/2605.20098/) (2026)
- [Pretrained LLMs Learn Multiple Types of Uncertainty](https://mlsanity.com/paper/2505.21218/) (2025)
- [Learning What to Predict: Downstream-Guided Task Design for Continued Pretraining](https://mlsanity.com/paper/2601.22108/) (2026)
- [`Generalization is hallucination' through the lens of tensor completions](https://mlsanity.com/paper/2502.17305/) (2025)
- [LaCy: What Small Language Models Can and Should Learn is Not Just a Question of Loss](https://mlsanity.com/paper/2602.12005/) (2026)
- [Three Concrete Challenges and Two Hopes for the Safety of Unsupervised Elicitation](https://mlsanity.com/paper/2602.20400/) (2026)
- [What Models Know, How Well They Know It: Knowledge-Weighted Fine-Tuning for Learning When to Say "I Don't Know"](https://mlsanity.com/paper/2604.05779/) (2026)
- [Leveraging Visual Signals for Robust Token-Level Uncertainty in Vision-Language Generation](https://mlsanity.com/paper/2605.27136/) (2026)
- [Knowledgeless Language Models: Suppressing Parametric Recall for Evidence-Grounded Language Modeling](https://mlsanity.com/paper/2607.12831/) (2026)
- [The Commit-Abstain Circuit: Why Language Models Hallucinate Instead of Abstaining](https://mlsanity.com/paper/2609.32964/) (2026)

## Abstract (arXiv)

Large Language Models are known to capture real-world knowledge, allowing them to excel in many downstream tasks. Despite recent advances, these models are still prone to what are commonly known as hallucinations, causing them to emit unwanted and factually incorrect text. In this work, we propose a novel calibration method that can be used to combat hallucinations. We add a special [IDK] ("I don't know") token to the model's vocabulary and introduce an objective function that shifts probability mass to the [IDK] token for incorrect predictions. This approach allows the model to express uncertainty in its output explicitly. We evaluate our proposed method across multiple model architectures and factual downstream tasks. We find that models trained with our method are able to express uncertainty in places where they would previously make mistakes while suffering only a small loss of encoded knowledge. We further perform extensive ablation studies of multiple variations of our approach and provide a detailed analysis of the precision-recall tradeoff of our method.