mlsanityData to 7 Oct 2026

Ask, Condition or Abstain: Reinforcement Learning for Missing-Premise Reasoning

arXiv 2608.16554 ↗First posted 17 Aug 2026cs.CLcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Answer-only reinforcement learning (RL) trains reasoning models to solve fully specified problems, but many realistic queries omit a premise needed for a unique answer. In this setting, the useful response is not always refusal: the model should ask for the missing premise, condition its answer on the unknown quantity, or abstain when no informative conditional response is available. We present Ask-Condition-Abstain Reinforcement Learning (ACA-RL), a data-augmented RL framework for this setting. Its reasoning-graph-guided pipeline converts well-posed problems into missing-premise training instances with localized gap annotations; ACA-RL then trains on these instances with a structured reward over five observable response behaviors. We also introduce the Missing-Premise Benchmark (MPB), a 274-instance human-verified benchmark spanning mathematical, logical, and real-world word problems. Across Qwen3 and Llama models, ACA-RL consistently improves on MPB while preserving competitive performance on well-posed reasoning tasks. Together with the released code, MPB, and training data, this work supports a new mission for NLP evaluation: measuring whether models can recognize when a task is underdetermined and handle uncertainty, not only whether they can answer fully specified questions.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 26 of its 33 references that are in the map. First posted on arXiv 17 Aug 2026.

Builds on

26 of 33 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Rethinking Reasoning Quality in Large Language Models through Enhanced Chain-of-Thought via RL7 Sep
  2. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Training22 Nov
  3. 2022Solving math word problems with process- and outcome-based feedback25 Nov
  4. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 22

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  4. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  5. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 385
  6. 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Modelscited by 246
  7. 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questionscited by 129
  8. 2024Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?cited by 85
  9. 2025Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinationscited by 48
  10. 2024Benchmarking Hallucination in Large Language Models based on Unanswerable Math Word Problemcited by 42
  11. 2024Uncertainty of Thoughts: Uncertainty-Aware Planning Enhances Information Seeking in Large Language Modelscited by 37
  12. 2025The Hallucination Tax of Reinforcement Finetuningcited by 35
10 more
  1. 2024BPO: Towards Balanced Preference Optimization between Knowledge Breadth and Depth in Alignmentcited by 20
  2. 2023Eliminating Reasoning via Inferring with Planning: A New Framework to Guide LLMs' Non-linear Thinkingcited by 19
  3. 2024Large Language Models Struggle with Unreasonability in Math Problemscited by 18
  4. 2024LLMs Know What They Need: Leveraging a Missing Information Guided Framework to Empower Retrieval-Augmented Generationcited by 17
  5. 2024Efficient Non-Parametric Uncertainty Quantification for Black-Box Large Language Models and Decision Planningcited by 16
  6. 2024Optimizing Language Model's Reasoning Abilities with Weak Supervisioncited by 15
  7. 2025TreeCut: A Synthetic Unanswerable Math Word Problem Dataset for LLM Hallucination Evaluationcited by 11
  8. 2025Beyond Accuracy: The Role of Calibration in Self-Improving Large Language Modelscited by 7
  9. 2025Entropy-Guided Loop: Achieving Reasoning through Uncertainty-Aware Generationcited by 2
  10. 2024LaMsS: When Large Language Models Meet Self-Skepticismcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Abstain-R1: Calibrated Abstention and Post-Refusal Clarification via Verifiable RL
  2. 2025Reinforcement Learning is all You Need
  3. 2025RAVR: Reference-Answer-guided Variational Reasoning for Large Language Models
  4. 2026Bridging the Detection-to-Abstention Gap in Reasoning Models under Insufficient Information
  5. 2026DARL: Encouraging Diverse Answers for General Reasoning without Verifiers
  6. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Data
  7. 2025You Need Reasoning to Learn Reasoning: The Limitations of Label-Free RL in Weak Base Models
  8. 2025Concise Reasoning via Reinforcement Learning