mlsanityData to 7 Oct 2026

When and What to Ask: AskBench and Rubric-Guided RLVR for LLM Clarification

arXiv 2602.11199 ↗First posted 4 Feb 2026cs.CL cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) often respond even when prompts omit critical details or include misleading information, leading to hallucinations or reinforced misconceptions. We study how to evaluate and improve LLMs' ability to decide when and what to ask for clarification without sacrificing task performance. We introduce AskBench, an interactive benchmark that converts standard QA pairs into multi-turn interactions with explicit checkpoints. A unified judge loop evaluates final answers and simulates user responses as needed. AskBench covers two settings: AskMind, with intent-deficient queries requiring clarification, and AskOverconfidence, with queries containing false premises that must be identified and corrected. We further propose rubric-guided reinforcement learning with verifier-based rewards (RLVR), which uses structured rubrics to encourage targeted clarification. Experiments show consistent improvements in accuracy, rubric adherence, and interaction efficiency, with strong generalization to unseen domains.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 27 of its 38 references that are in the map. First posted on arXiv 4 Feb 2026.

Builds on

27 of 38 references.

References · 27

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  4. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  5. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  6. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  7. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  8. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  9. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  10. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  11. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  12. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
15 more
  1. 2019Asking Clarifying Questions in Open-Domain Information-Seeking Conversationscited by 136
  2. 2018Interpretation of Natural Language Rules in Conversational Machine Readingcited by 72
  3. 2024Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agentscited by 71
  4. 2024API Is Enough: Conformal Prediction for Large Language Models Without Logit-Accesscited by 68
  5. 2025QuestBench: Can LLMs ask the right question to acquire information in reasoning tasks?cited by 56
  6. 2024Reasoning-Enhanced Healthcare Predictions with Knowledge Graph Community Retrievalcited by 45
  7. 2026RubricHub: A Comprehensive and Highly Discriminative Rubric Dataset via Automated Coarse-to-Fine Generationcited by 42
  8. 2023Teach AI How to Code: Using Large Language Models as Teachable Agents for Programming Educationcited by 38
  9. 2020ClarQ: A large-scale and diverse dataset for Clarification Question Generationcited by 31
  10. 2024The RealHumanEval: Evaluating Large Language Models' Abilities to Support Programmerscited by 28
  11. 2025How to Protect Yourself from 5G Radiation? Investigating LLM Responses to Implicit Misinformationcited by 17
  12. 2025AskToAct: Enhancing LLMs Tool Use via Self-Correcting Clarificationcited by 12
  13. 2021Open-Retrieval Conversational Machine Readingcited by 11
  14. 2025First Ask Then Answer: A Framework Design for AI Dialogue Based on Supplementary Questioning with Large Language Modelscited by 2
  15. 2025Robust Uncertainty Quantification for Self-Evolving Large Language Models via Continual Domain Pretrainingcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 12.

12 papers.

  1. 2026AgentAbstain: Do LLM Agents Know When Not to Act?cited by 8
  2. 2026Benchmarking Multi-turn Medical Diagnosis: Hold, Lure, and Self-Correctioncited by 4
  3. 2026From Holistic Evaluation to Structured Criteria: Rubrics Across the Evolving LLM Landscapecited by 4
  4. 2026Ambig-DS: A Benchmark for Task-Framing Ambiguity in Data-Science Agentscited by 2
  5. 2026Don't Start What You Can't Finish: A Counterfactual Audit of Support-State Triage in LLM Agentscited by 0
  6. 2026MIRA-Math: A Benchmark for Minimal Information Requesting and Mathematical Reasoningcited by 0
  7. 2026FinRiskAtlas: Decision-Aligned Evaluation of Large Language Models for Financial Risk Reviewcited by 0
  8. 2026Ask Before You Optimize: Dynamic Pre-Formulation Clarification for Interactive Optimizationcited by 0
  9. 2026When and How Should an Agent Clarify? CIGAsk: Teaching LLMs to Clarify via Counterfactual Information Gaincited by 0
  10. 2026Clarification Is Not Correction: LLMs Fail to Let Gocited by 0
  11. 2026Asking for What Was Never Requested: Horizontal and Vertical Proactivity in Agentscited by 0
  12. 2026Learning When and How to Intervene: A Hindsight-Distilled Sentinel for Coding Agentscited by 0