mlsanityData to 7 Oct 2026

CoDiQ: Test-Time Scaling for Controllable Difficult Question Generation

arXiv 2602.01660 ↗First posted 2 Feb 2026cs.CL cs.AIcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large Reasoning Models (LRMs) benefit substantially from training on challenging competition-level questions. However, existing automated question synthesis methods lack precise difficulty control, incur high computational costs, and struggle to generate competition-level questions at scale. In this paper, we propose CoDiQ (Controllable Difficult Question Generation), a novel framework enabling fine-grained difficulty control via test-time scaling while ensuring question solvability. Specifically, first, we identify a test-time scaling tendency (extended reasoning token budget boosts difficulty but reduces solvability) and the intrinsic properties defining the upper bound of a model's ability to generate valid, high-difficulty questions. Then, we develop CoDiQ-Generator from Qwen3-8B, which improves the upper bound of difficult question generation, making it particularly well-suited for challenging question construction. Building on the CoDiQ framework, we build CoDiQ-Corpus (44K competition-grade question sequences). Human evaluations show these questions are significantly more challenging than LiveCodeBench/AIME with over 82% solvability. Training LRMs on CoDiQ-Corpus substantially improves reasoning performance, verifying that scaling controlled-difficulty training questions enhances reasoning capabilities. We open-source CoDiQ-Corpus, CoDiQ-Generator, and implementations to support related research.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 27 of its 34 references that are in the map. First posted on arXiv 2 Feb 2026.

Builds on

27 of 34 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Qwen3 Technical Report14 May
  2. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools18 Jun

Other references · 25

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2021Program Synthesis with Large Language Modelscited by 3,509
  4. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  5. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  6. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  7. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
  8. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  9. 2021Measuring Coding Challenge Competence With APPScited by 879
  10. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 442
  11. 2025LeetCodeDataset: A Temporal Dataset for Robust Evaluation and Efficient Training of Code LLMscited by 67
  12. 2025Challenging the Boundaries of Reasoning: An Olympiad-Level Math Benchmark for Large Language Modelscited by 65
13 more
  1. 2025On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Modelscited by 64
  2. 2024Augmenting Math Word Problems via Iterative Question Composingcited by 58
  3. 2024AI-Assisted Generation of Difficult Math Questionscited by 37
  4. 2025The LLM Already Knows: Estimating LLM-Perceived Question Difficulty via Hidden Representationscited by 26
  5. 2024Unleashing LLM Reasoning Capability via Scalable Question Synthesis from Scratchcited by 24
  6. 2025SAND-Math: Using LLMs to Generate Novel, Difficult and Useful Mathematics Questions and Answerscited by 11
  7. 2025PromptCoT 2.0: Scaling Prompt Synthesis for Large Language Model Reasoningcited by 7
  8. 2024Adversarial Math Word Problem Generationcited by 6
  9. 2025ScaleDiff: Scaling Difficult Problems for Advanced Mathematical Reasoningcited by 6
  10. 2025QueST: Incentivizing LLMs to Generate Difficult Problemscited by 4
  11. 2025EvolMathEval: Towards Evolvable Benchmarks for Mathematical Reasoning via Evolutionary Testingcited by 3
  12. 2025CogAtom: From Cognitive Atoms to Olympiad-level Mathematical Reasoning in Large Language Modelscited by 2
  13. 2025UniCode: Augmenting Evaluation for Code Reasoningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Benchmarking Critical Questions Generation: A Challenging Reasoning Task for Large Language Models
  2. 2025T2: An Adaptive Test-Time Scaling Strategy for Contextual Question Answering
  3. 2025CCQA: Generating Question from Solution Can Improve Inference-Time Reasoning in SLMs
  4. 2026TTCS: Test-Time Curriculum Synthesis for Self-Evolving
  5. 2025Generate-Then-Validate: A Novel Question Generation Approach Using Small Language Models
  6. 2025Reasoning and Sampling-Augmented MCQ Difficulty Prediction via LLMs
  7. 2026KNIGHT: Knowledge Graph-Driven Multiple-Choice Question Generation with Adaptive Hardness Calibration
  8. 2025SynAdapt: Learning Adaptive Reasoning in Large Language Models via Synthetic Continuous Chain-of-Thought