mlsanityData to 14 Oct 2026

QFFT, Question-Free Fine-Tuning for Adaptive Reasoning

arXiv 2506.12860 ↗First posted 15 Jun 2025cs.CLcited by 15

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Recent advancements in Long Chain-of-Thought (CoT) reasoning models have improved performance on complex tasks, but they suffer from overthinking, which generates redundant reasoning steps, especially for simple questions. This paper revisits the reasoning patterns of Long and Short CoT models, observing that the Short CoT patterns offer concise reasoning efficiently, while the Long CoT patterns excel in challenging scenarios where the Short CoT patterns struggle. To enable models to leverage both patterns, we propose Question-Free Fine-Tuning (QFFT), a fine-tuning approach that removes the input question during training and learns exclusively from Long CoT responses. This approach enables the model to adaptively employ both reasoning patterns: it prioritizes the Short CoT patterns and activates the Long CoT patterns only when necessary. Experiments on various mathematical datasets demonstrate that QFFT reduces average response length by more than 50%, while achieving performance comparable to Supervised Fine-Tuning (SFT). Additionally, QFFT exhibits superior performance compared to SFT in noisy, out-of-domain, and low-resource scenarios.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 38 of its 50 references that are in the map. First posted on arXiv 15 Jun 2025.

Builds on

38 of 50 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025Unlocking Efficient Long-to-Short LLM Reasoning with Model Merging26 Mar
  2. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning6 Mar
  3. 2025TokenSkip: Controllable Chain-of-Thought Compression in LLMs17 Feb
  4. 2025CoT-Valve: Length-Compressible Chain-of-Thought Tuning13 Feb
  5. 2025Demystifying Long Chain-of-Thought Reasoning in LLMs5 Feb
  6. 2025Competitive Programming with Large Reasoning Models3 Feb
  7. 2025s1: Simple test-time scaling31 Jan
  8. 2025Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs30 Jan
  9. 2025O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning22 Jan
  10. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs30 Dec
  11. 2024Token-Budget-Aware LLM Reasoning24 Dec
  12. 2024Concise Thoughts: Impact of Output Length on LLM Reasoning and Cost29 Jul

Other references · 26

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2024Qwen2.5 Technical Reportcited by 4,942
  3. 2023Let's Verify Step by Stepcited by 3,789
  4. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  5. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  6. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  7. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  8. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  9. 2019A Comprehensive Survey on Transfer Learningcited by 1,160
  10. 2025LIMO: Less is More for Reasoningcited by 498
  11. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 377
  12. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 320
14 more
  1. 2025Chain of Draft: Thinking Faster by Writing Lesscited by 297
  2. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 293
  3. 2024ProcessBench: Identifying Process Errors in Mathematical Reasoningcited by 292
  4. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 257
  5. 2024Data Engineering for Scaling Language Models to 128K Contextcited by 186
  6. 2025AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning datasetcited by 174
  7. 2023Understanding Catastrophic Forgetting in Language Models via Implicit Inferencecited by 144
  8. 2025LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!cited by 132
  9. 2025Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thoughtcited by 109
  10. 2025Code to Think, Think to Code: A Survey on Code-Enhanced Reasoning and Reasoning-Driven Code Intelligence in LLMscited by 47
  11. 2024LongRecipe: Recipe for Efficient Long Context Generalization in Large Language Modelscited by 23
  12. 2025Reasoning on a Spectrum: Aligning LLMs to System 1 and System 2 Thinkingcited by 16
  13. 2025Marco-o1 v2: Towards Widening The Distillation Bottleneck for Reasoning Modelscited by 13
  14. 2025Enhancing Customer Contact Efficiency with Graph Neural Networks in Credit Card Fraud Detection Workflowcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 11; 2026: 4.

15 papers.

Sort:

By area:

  1. 2025HUD: Hierarchical Uncertainty-Aware Disambiguation Network for Composed Video Retrievalcited by 32
  2. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
  3. 2025AMAS: Adaptively Determining Communication Topology for LLM-based Multi-Agent Systemcited by 15
  4. 2025RaCoT: Plug-and-Play Contrastive Example Generation Mechanism for Enhanced LLM Reasoning Reliabilitycited by 14
  5. 2025FT-MDT: Extracting Decision Trees from Medical Texts via a Novel Low-rank Adaptation Methodcited by 7
  6. 2025Mixture of Length and Pruning Experts for Knowledge Graphs Reasoningcited by 5
  7. 2025Detecting Distillation Data from Reasoning Modelscited by 4
  8. 2025The Zero-Step Thinking: An Empirical Study of Mode Selection as Harder Early Exit in Reasoning Modelscited by 4
  9. 2026R2-Write: Reflection and Revision for Open-Ended Writing with Deep Reasoningcited by 3
  10. 2026From P(y|x) to P(y): Investigating Reinforcement Learning in Pre-train Spacecited by 3
  11. 2026SAT: Balancing Reasoning Accuracy and Efficiency with Stepwise Adaptive Thinkingcited by 2
  12. 2025Coarse-to-Fine Personalized LLM Impressions for Streamlined Radiology Reportscited by 1
3 more
  1. 2025Beyond Function-Level Search: Repository-Aware Dual-Encoder Code Retrieval with Adversarial Verificationcited by 1
  2. 2025ToM: Leveraging Tree-oriented MapReduce for Long-Context Reasoning in Large Language Modelscited by 1
  3. 2026Nice Fold or Hero Call: Learning Budget-Efficient Thinking for Adaptive Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning Eliciting Efficient Reasoning in Large Language Models
  2. 2025From Reasoning to Super-Intelligence: A Search-Theoretic Perspective
  3. 2025Rethinking Supervised Fine-Tuning: Emphasizing Key Answer Tokens for Improved LLM Accuracy
  4. 2025Thinking Preference Optimization
  5. 2025Ada-R1: Hybrid-CoT via Bi-Level Adaptive Reasoning Optimization
  6. 2024ReFT: Reasoning with Reinforced Fine-Tuning
  7. 2026Zipping the Thought: When and How Compressed Reasoning Data Works in LLM Post-Training
  8. 2026Correct, Concise and Complete: Multi-stage Training For Adaptive Reasoning