# Beyond Passive Critical Thinking: Fostering Proactive Questioning to Enhance Human-AI Collaboration

arXiv: https://arxiv.org/abs/2507.23407 · HTML: https://mlsanity.com/paper/2507.23407/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 30 references that are in the map.
- First posted on arXiv 31 Jul 2025.

## Builds on (most cited first, 23 of 23)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solvers](https://mlsanity.com/paper/2402.19255/) (2024)
- [AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions](https://mlsanity.com/paper/2506.09038/) (2025)
- [CollabLLM: From Passive Responders to Active Collaborators](https://mlsanity.com/paper/2502.00640/) (2025)
- [STaR-GATE: Teaching Language Models to Ask Clarifying Questions](https://mlsanity.com/paper/2403.19154/) (2024)
- [ISR-LLM: Iterative Self-Refined Large Language Model for Long-Horizon Sequential Task Planning](https://mlsanity.com/paper/2308.13724/) (2023)
- [Modeling Future Conversation Turns to Teach LLMs to Ask Clarifying Questions](https://mlsanity.com/paper/2410.13788/) (2024)
- [CLAM: Selective Clarification for Ambiguous Questions with Generative Language Models](https://mlsanity.com/paper/2212.07769/) (2022)
- [Missing Premise exacerbates Overthinking: Are Reasoning Models losing Critical Thinking Skill?](https://mlsanity.com/paper/2504.06514/) (2025)
- [Benchmarking Hallucination in Large Language Models based on Unanswerable Math Word Problem](https://mlsanity.com/paper/2403.03558/) (2024)
- [Learning to Ask: When LLM Agents Meet Unclear Instruction](https://mlsanity.com/paper/2409.00557/) (2024)
- [The Hallucination Tax of Reinforcement Finetuning](https://mlsanity.com/paper/2505.13988/) (2025)
- [Understanding Users' Dissatisfaction with ChatGPT Responses: Types, Resolving Tactics, and the Effect of Knowledge Level](https://mlsanity.com/paper/2311.07434/) (2023)
- [ALFA: Aligning LLMs to Ask Good Questions A Case Study in Clinical Reasoning](https://mlsanity.com/paper/2502.14860/) (2025)
- [EpiCoder: Encompassing Diversity and Complexity in Code Generation](https://mlsanity.com/paper/2501.04694/) (2025)
- [Large Language Models Struggle with Unreasonability in Math Problems](https://mlsanity.com/paper/2403.19346/) (2024)
- [LPML: LLM-Prompting Markup Language for Mathematical Reasoning](https://mlsanity.com/paper/2309.13078/) (2023)
- [Investigating Inference-time Scaling for Chain of Multi-modal Thought: A Preliminary Study](https://mlsanity.com/paper/2502.11514/) (2025)
- [From Blind Solvers to Logical Thinkers: Benchmarking LLMs' Logical Integrity on Faulty Mathematical Problems](https://mlsanity.com/paper/2410.18921/) (2024)
- [UniGenCoder: Merging Seq2Seq and Seq2Tree Paradigms for Unified Code Generation](https://mlsanity.com/paper/2502.12490/) (2025)

## Built on by (most cited first, 5 of 5)

- [Abstain-R1: Calibrated Abstention and Post-Refusal Clarification via Verifiable RL](https://mlsanity.com/paper/2604.17073/) (2026)
- [Statistical Early Stopping for Reasoning Models](https://mlsanity.com/paper/2602.13935/) (2026)
- [Knowing Isn't Understanding: Re-grounding Generative Proactivity with Epistemic and Behavioral Insight](https://mlsanity.com/paper/2602.15259/) (2026)
- [MMPCBench: Benchmarking Multimodal Large Language Models on Proactive Critique of Flawed Inputs](https://mlsanity.com/paper/2608.29286/) (2026)
- [Rewarding Efficient Reasoning Improves Abstention on Underspecified Tasks in Reasoning Models](https://mlsanity.com/paper/2609.20846/) (2026)

## Abstract (arXiv)

Critical thinking is essential for building robust AI systems, preventing them from blindly accepting flawed data or biased reasoning. However, prior work has primarily focused on passive critical thinking, where models simply reject problematic queries without taking constructive steps to address user requests. In this work, we introduce proactive critical thinking, a paradigm where models actively seek missing or clarifying information from users to resolve their queries better. To evaluate this capability, we present GSM-MC and GSM-MCE, two novel benchmarks based on GSM8K for assessing mathematical reasoning under incomplete or misleading conditions. GSM-MC contains 1,368 math problems with a key variable deliberately removed, requiring models to identify and request the missing information. GSM-MCE further increases the difficulty by introducing irrelevant details to test robustness against distractions. Experiments on Qwen3 and Llama series models show that, while these models excel in traditional reasoning tasks due to extensive post-training and inference-time scaling, they struggle with proactive critical thinking, especially smaller ones. However, we demonstrate that reinforcement learning (RL) can significantly improve this ability. Using our enhanced RL algorithm, we achieve substantial gains, boosting the Qwen3-1.7B's accuracy from 0.15% to 73.98% on GSM-MC. We hope this work advances models that collaborate more effectively with users in problem-solving through proactive critical thinking.