mlsanityData to 14 Oct 2026

CCQA: Generating Question from Solution Can Improve Inference-Time Reasoning in SLMs

arXiv 2509.18536 ↗First posted 23 Sep 2025cs.CL cs.AIcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Recently, inference-time reasoning strategies have further improved the accuracy of large language models (LLMs), but their effectiveness on smaller models remains unclear. Based on the observation that conventional approaches often fail to improve performance in this context, we propose Cycle-Consistency in Question Answering (CCQA), a novel reasoning method that can be effectively applied to SLMs. Inspired by cycle consistency, CCQA generates a question from each reasoning path and answer, evaluates each by its similarity to the original question, and then selects the candidate solution with the highest similarity score as the final response. Since conventional SLMs struggle to generate accurate questions from their own reasoning paths and answers, we employ a lightweight Flan-T5 model specialized for question generation to support this process efficiently. From the experimental results, it is verified that CCQA consistently outperforms existing state-of-the-art (SOTA) methods across eight models on mathematical and commonsense reasoning benchmarks. Furthermore, our method establishes a new practical baseline for efficient reasoning in SLMs. Source code can be found at https://github.com/scai-research/ccqaofficial.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 32 of its 38 references that are in the map. First posted on arXiv 23 Sep 2025.

Builds on

32 of 38 references.

References · 32

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  8. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  9. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  10. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  11. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  12. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
20 more
  1. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
  2. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  3. 2017CyCADA: Cycle-Consistent Adversarial Domain Adaptationcited by 1,082
  4. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  5. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
  6. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 788
  7. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  8. 2016Solving General Arithmetic Word Problemscited by 360
  9. 2024A Survey on Data Selection for Language Modelscited by 236
  10. 2024MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Casescited by 223
  11. 2023Universal Self-Consistency for Large Language Model Generationcited by 203
  12. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 159
  13. 2016Dual Learning for Machine Translationcited by 148
  14. 2024Mobile Edge Intelligence for Large Language Models: A Contemporary Surveycited by 128
  15. 2019Neural Machine Translation: A Review and Surveycited by 100
  16. 2024Why Does the Effective Context Length of LLMs Fall Short?cited by 63
  17. 2018Multi-view Consistency as Supervisory Signal for Learning Shape and Pose Predictioncited by 52
  18. 2024Why Larger Language Models Do In-context Learning Differently?cited by 45
  19. 2022Commonsense Knowledge Reasoning and Generation with Pre-trained Language Models: A Surveycited by 38
  20. 2024Scaling Parameter-Constrained Language Models with Quality Datacited by 12

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Question Decomposition Improves the Faithfulness of Model-Generated Reasoning
  2. 2025ReTraceQA: Evaluating Reasoning Traces of Small Language Models in Commonsense Question Answering
  3. 2025T2: An Adaptive Test-Time Scaling Strategy for Contextual Question Answering
  4. 2022Faithful Reasoning Using Large Language Models
  5. 2024Reasoning Aware Self-Consistency: Leveraging Reasoning Paths for Efficient LLM Sampling
  6. 2024MINTQA: A Multi-Hop Question Answering Benchmark for Evaluating LLMs on New and Tail Knowledge
  7. 2026Tandem: Riding Together with Large and Small Language Models for Efficient Reasoning
  8. 2024CLR-Bench: Evaluating Large Language Models in College-level Reasoning