# Knowledge Boundary Discovery for Large Language Models

arXiv: https://arxiv.org/abs/2603.21022 · HTML: https://mlsanity.com/paper/2603.21022/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 28 references that are in the map.
- First posted on arXiv 14 Jan 2026.

## Builds on (most cited first, 20 of 20)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools](https://mlsanity.com/paper/2406.12793/) (2024)
- [On Faithfulness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2005.00661/) (2020)
- [The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only](https://mlsanity.com/paper/2306.01116/) (2023)
- [INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection](https://mlsanity.com/paper/2402.03744/) (2024)
- [Fine-tuning Language Models for Factuality](https://mlsanity.com/paper/2311.08401/) (2023)
- [LM vs LM: Detecting Factual Errors via Cross Examination](https://mlsanity.com/paper/2305.13281/) (2023)
- [Extreme Q-Learning: MaxEnt RL without Entropy](https://mlsanity.com/paper/2301.02328/) (2023)
- [How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions](https://mlsanity.com/paper/2309.15840/) (2023)
- [Self-Alignment for Factuality: Mitigating Hallucinations in LLMs via Self-Evaluation](https://mlsanity.com/paper/2402.09267/) (2024)
- [Benchmarking Knowledge Boundary for Large Language Models: A Different Perspective on Model Evaluation](https://mlsanity.com/paper/2402.11493/) (2024)
- [Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learning](https://mlsanity.com/paper/2205.14704/) (2022)
- [Examining LLMs' Uncertainty Expression Towards Questions Outside Parametric Knowledge](https://mlsanity.com/paper/2311.09731/) (2023)
- [Teaching Large Language Models to Express Knowledge Boundary from Their Own Signals](https://mlsanity.com/paper/2406.10881/) (2024)
- [Measuring and Modifying Factual Knowledge in Large Language Models](https://mlsanity.com/paper/2306.06264/) (2023)
- [Asking for Knowledge: Training RL Agents to Query External Knowledge Using Language](https://mlsanity.com/paper/2205.06111/) (2022)
- [Model-free Posterior Sampling via Learning Rate Randomization](https://mlsanity.com/paper/2310.18186/) (2023)
- [Goal-Conditioned Q-Learning as Knowledge Distillation](https://mlsanity.com/paper/2208.13298/) (2022)

## Built on by (most cited first, 1 of 1)

- [KBF: Knowledge Boundary as Fingerprint for Language Model and Black-Box API Auditing](https://mlsanity.com/paper/2605.29524/) (2026)

## Abstract (arXiv)

We propose Knowledge Boundary Discovery (KBD), a reinforcement learning based framework to explore the knowledge boundaries of the Large Language Models (LLMs). We define the knowledge boundary by automatically generating two types of questions: (i) those the LLM can confidently answer (within-knowledge boundary) and (ii) those it cannot (beyond-knowledge boundary). Iteratively exploring and exploiting the LLM's responses to find its knowledge boundaries is challenging because of the hallucination phenomenon. To find the knowledge boundaries of an LLM, the agent interacts with the LLM under the modeling of exploring a partially observable environment. The agent generates a progressive question as the action, adopts an entropy reduction as the reward, receives the LLM's response as the observation and updates its belief states. We demonstrate that the KBD detects knowledge boundaries of LLMs by automatically finding a set of non-trivial answerable and unanswerable questions. We validate the KBD by comparing its generated knowledge boundaries with manually crafted LLM benchmark datasets. Experiments show that our KBD-generated question set is comparable to the human-generated datasets. Our approach paves a new way to evaluate LLMs.