mlsanityData to 14 Oct 2026

Don't Just Say "I don't know"! Self-aligning Large Language Models for Responding to Unknown Questions with Explanations

arXiv 2402.15062 ↗First posted 23 Feb 2024cs.CL cs.LGcited by 30

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

Despite the remarkable abilities of Large Language Models (LLMs) to answer questions, they often display a considerable level of overconfidence even when the question does not have a definitive answer. To avoid providing hallucinated answers to these unknown questions, existing studies typically investigate approaches to refusing to answer these questions. In this work, we propose a novel and scalable self-alignment method to utilize the LLM itself to enhance its response-ability to different types of unknown questions, being capable of not only refusing to answer but also providing explanation to the unanswerability of unknown questions. Specifically, the Self-Align method first employ a two-stage class-aware self-augmentation approach to generate a large amount of unknown question-response data. Then we conduct disparity-driven self-curation to select qualified data for fine-tuning the LLM itself for aligning the responses to unknown questions as desired. Experimental results on two datasets across four types of unknown questions validate the superiority of the Self-Align method over existing baselines in terms of three types of task formulation.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 27 of its 45 references that are in the map. First posted on arXiv 23 Feb 2024.

Builds on

27 of 45 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback24 May
  2. 2023Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision4 May
  3. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructions20 Dec

Other references · 24

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  4. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  5. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  6. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  7. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  8. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
  9. 2015Teaching Machines to Read and Comprehendcited by 1,049
  10. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  11. 2022Large Language Models Can Self-Improvecited by 664
  12. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 390
12 more
  1. 2022Prompting GPT-3 To Be Reliablecited by 260
  2. 2023Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaborationcited by 116
  3. 2023Decomposing Uncertainty for Large Language Models through Input Clarification Ensemblingcited by 114
  4. 2023A Survey on Proactive Dialogue Systems: Problems, Methods, and Prospectscited by 66
  5. 2023Cue-CoT: Chain-of-thought Prompting for Responding to In-depth Dialogue Questions with LLMscited by 62
  6. 2019TEQUILA: Temporal Question Answering over Knowledge Basescited by 47
  7. 2023Beyond Factuality: A Comprehensive Evaluation of Large Language Models as Knowledge Generatorscited by 43
  8. 2021Exploiting Reasoning Chains for Multi-hop Science Question Answeringcited by 15
  9. 2023SCENE: Self-Labeled Counterfactuals for Extrapolating to Negative Examplescited by 8
  10. 2021Contextualized Knowledge-aware Attentive Neural Network: Enhancing Answer Selection with Knowledgecited by 7
  11. 2022Context-Situated Pun Generationcited by 5
  12. 2023Can NLP Models 'Identify', 'Distinguish', and 'Justify' Questions that Don't have a Definitive Answer?cited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 17; 2026: 7.

30 papers.

Sort:

By area:

  1. 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questionscited by 145
  2. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 121
  3. 2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directionscited by 50
  4. 2024Knowledge Boundary of Large Language Models: A Surveycited by 40
  5. 2024Perception of Knowledge Boundary for Large Language Models through Semi-open-ended Question Answeringcited by 20
  6. 2024Cannot or Should Not? Automatic Analysis of Refusal Composition in IFT/RLHF Datasets and Refusal Behavior of Black-Box LLMscited by 11
  7. 2025Do not Abstain! Identify and Solve the Uncertaintycited by 11
  8. 2025BARREL: Boundary-Aware Reasoning for Factual and Reliable LRMscited by 8
  9. 2025Line of Duty: Evaluating LLM Self-Knowledge via Consistency in Feasibility Boundariescited by 7
  10. 2025Can LLMs Estimate Student Struggles? Human-AI Difficulty Alignment with Proficiency Simulation for Item Difficulty Predictioncited by 7
  11. 2025Fast or Better? Balancing Accuracy and Cost in Retrieval-Augmented Generation with Flexible User Controlcited by 6
  12. 2025Mitigating Lost in Multi-turn Conversation via Curriculum RL with Verifiable Accuracy and Abstention Rewardscited by 6
18 more
  1. 2025SHARP: Synthesizing High-quality Aligned Reasoning Problems for Large Reasoning Models Reinforcement Learningcited by 5
  2. 2025Do Retrieval Augmented Language Models Know When They Don't Know?cited by 5
  3. 2024Tell Me What You Don't Know: Enhancing Refusal Capabilities of Role-Playing Agents via Representation Space Analysis and Editingcited by 3
  4. 2025When Bias Pretends to Be Truth: How Spurious Correlations Undermine Hallucination Detection in LLMscited by 3
  5. 2025Knowing What's Missing: Assessing Information Sufficiency in Question Answeringcited by 3
  6. 2024When Not to Answer: Evaluating Prompts on GPT Models for Effective Abstention in Unanswerable Math Word Problemscited by 2
  7. 2025LLM Context Conditioning and PWP Prompting for Multimodal Validation of Chemical Formulascited by 2
  8. 2025VLM@school -- Evaluation of AI image understanding on German middle school knowledgecited by 2
  9. 2025A DbC Inspired Neurosymbolic Layer for Trustworthy Agent Designcited by 2
  10. 2026PROPER Agents: Proactivity Driven Personalized Agents for Advancing Knowledge Gap Navigationcited by 2
  11. 2026Beyond "I Don't Know": Evaluating LLM Self-Awareness in Discriminating Data and Model Uncertaintycited by 2
  12. 2026SCOPE: Sequential Conformal Probing for Reliable OOD Rejection in LLM Servicescited by 2
  13. 2025FactGuard: Leveraging Multi-Agent Systems to Generate Answerable and Unanswerable Questions for Enhanced Long-Context LLM Extractioncited by 1
  14. 2026Learning from AVA: Early Lessons from a Curated and Trustworthy Generative AI for Policy and Development Researchcited by 1
  15. 2025The Idola Tribus of AI: Large Language Models tend to perceive order where none existscited by 0
  16. 2026"I Don't Know" -- Towards Appropriate Trust with Certainty-Aware Retrieval Augmented Generationcited by 0
  17. 2026Automatically Evolving Prompt Guidelines for Task-Specific Optimizationcited by 0
  18. 2026Rewarding Efficient Reasoning Improves Abstention on Underspecified Tasks in Reasoning Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025CALM: Unleashing the Cross-Lingual Self-Aligning Ability of Language Model Question Answering
  2. 2025No Answer Needed: Predicting LLM Answer Accuracy from Question-Only Linear Probes
  3. 2024Think Twice Before Trusting: Self-Detection for Large Language Models through Comprehensive Answer Reflection
  4. 2023Self-Consistency of Large Language Models under Ambiguity
  5. 2023Knowledge of Knowledge: Exploring Known-Unknowns Uncertainty with Large Language Models
  6. 2025Collective Reasoning Among LLMs: A Framework for Answer Validation Without Ground Truth
  7. 2023Self-QA: Unsupervised Knowledge Guided Language Model Alignment
  8. 2022Discovering Latent Knowledge in Language Models Without Supervision