mlsanityData to 14 Oct 2026

Learning to Reason Across Parallel Samples for LLM Reasoning

arXiv 2506.09014 ↗First posted 10 Jun 2025cs.CLcited by 32

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Scaling test-time compute brings substantial performance gains for large language models (LLMs). By sampling multiple answers and heuristically aggregate their answers (e.g., either through majority voting or using verifiers to rank the answers), one can achieve consistent performance gains in math domains. In this paper, we propose a new way to leverage such multiple sample set. We train a compact LLM, called Sample Set Aggregator (SSA), that takes a concatenated sequence of multiple samples and output the final answer, optimizing it for the answer accuracy with reinforcement learning. Experiments on five reasoning datasets demonstrate both the efficacy and efficiency of SSA. Notably, SSA improves over naive majority voting by 8% pass@5 on MATH. Furthermore, our 3B SSA surpasses model-based re-ranking with a much larger 72B process reward model. Our analysis also shows promising generalization ability of SSA, across sample set sizes, base model families and scales, and tasks. By separating LLMs to generate answers and LLMs to analyze and aggregate sampled answers, our approach can work with the outputs from premier black box models easily and efficiently.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 28 of its 66 references that are in the map. First posted on arXiv 10 Jun 2025.

Builds on

28 of 66 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning6 Mar
  2. 2025Revisiting the Test-Time Scaling of o1-like Models: Do they Truly Possess Test-Time Scaling Capabilities?17 Feb
  3. 2025s1: Simple test-time scaling31 Jan
  4. 2024Soft Self-Consistency Improves Language Model Agents20 Feb

Other references · 24

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2025Qwen3 Technical Reportcited by 8,820
  6. 2020Scaling Laws for Neural Language Modelscited by 6,586
  7. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  8. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  9. 2024Qwen2.5 Technical Reportcited by 4,942
  10. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  11. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  12. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
12 more
  1. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  2. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  3. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  4. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  5. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 625
  6. 2023Textbooks Are All You Needcited by 569
  7. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 511
  8. 2025The Lessons of Developing Process Reward Models in Mathematical Reasoningcited by 415
  9. 2024Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learningcited by 241
  10. 2025A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibilitycited by 93
  11. 2025Organize the Web: Constructing Domains Enhances Pre-Training Data Curationcited by 76
  12. 2025Enhancing Reasoning through Process Supervision with Monte Carlo Tree Searchcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 11; 2026: 21.

32 papers.

Sort:

By area:

  1. 2025The Majority is not always right: RL training for solution aggregationcited by 47
  2. 2026Agentic Aggregation for Parallel Scaling of Long-Horizon Agentic Taskscited by 18
  3. 2025A Survey on Parallel Reasoningcited by 11
  4. 2025Small Drafts, Big Verdict: Information-Intensive Visual Reasoning via Speculationcited by 8
  5. 2025Making, not Taking, the Best of Ncited by 6
  6. 2025Generalized Parallel Scaling with Interdependent Generationscited by 6
  7. 2025Think in Parallel, Answer as One: Logit Averaging for Open-Ended Reasoningcited by 6
  8. 2025AlphaApollo: A System for Deep Agentic Reasoningcited by 5
  9. 2025Off-Trajectory Reasoning: Can LLMs Collaborate on Reasoning Trajectory?cited by 5
  10. 2025Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanismcited by 4
  11. 2026Online Reasoning Calibration: Test-Time Training Enables Generalizable Conformal LLM Reasoningcited by 4
  12. 2026OpenDeepThink: Parallel Reasoning via Bradley-Terry Aggregationcited by 3
20 more
  1. 2026AgentFugue: Agent Scaling for Long-Horizon Tasks through Collective Reasoningcited by 3
  2. 2026Multi-Agent Computer Usecited by 3
  3. 2026SPIRAL: Learning to Search and Aggregatecited by 3
  4. 2026ELASTIC: Efficiently Learning to Adaptively Scale Test-Time Compute for Generative Control Policiescited by 3
  5. 2026Learning Generative Selection for Best-of-Ncited by 2
  6. 2026LLMs Know When They Know, but Do Not Act on It: A Metacognitive Harness for Test-time Scalingcited by 2
  7. 2025A2R: An Asymmetric Two-Stage Reasoning Framework for Parallel Reasoningcited by 1
  8. 2025Critique to Verify: Accurate and Honest Test-Time Scaling with RL-Trained Verifierscited by 1
  9. 2026Learning from Self-Debate: Preparing Reasoning Models for Multi-Agent Debatecited by 1
  10. 2026Divide-and-Conquer CoT: RL for Reducing Latency via Parallel Reasoningcited by 1
  11. 2026Adaptive Ability Decomposing for Unlocking Large Reasoning Model Effective Reinforcement Learningcited by 1
  12. 2026LaneRoPE: Positional Encoding for Collaborative Parallel Reasoning and Generationcited by 1
  13. 2026FineVerify: Scaling Test-Time Compute with Fine-Grained Self-Verification for Agentic Searchcited by 1
  14. 2026Diverse Thinking Schemata Elicit Better Reasoning in Large Language Modelscited by 1
  15. 2026Stage-Replay Divergence Follows the KV Cache: Fixed-Prefix Precision Controls and Bidirectional Cache Transplantationcited by 1
  16. 2026A Single Revision Step Improves Token-Efficient LLM Reasoningcited by 0
  17. 2026Efficient Reasoning on the Edgecited by 0
  18. 2026Toward Skill-Native LLMs: Skill Entropy for Benchmarking and Training Long-Horizon Reasoningcited by 0
  19. 2026Selection, Recombination, or a Fresh Solve? A Candidate-Free Control for Single-Pass Test-Time Aggregationcited by 0
  20. 2026Diagnosing Sampled LLM Reasoning in Formal Geometry: Coverage, Realization, and Validity Evidencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Refining Over Resampling: Test-Time Self-Correction for LLM Reasoning
  2. 2025HS-STaR: Hierarchical Sampling for Self-Taught Reasoners via Difficulty Estimation and Budget Reallocation
  3. 2026Divergent-Convergent Reasoning: Scaling Test-Time Compute through Structured Solution Synthesis
  4. 2024Refining Answer Distributions for Improved Large Language Model Reasoning
  5. 2025Learning to Reason in LLMs by Expectation Maximization
  6. 2025Incentivizing LLMs to Self-Verify Their Answers
  7. 2026Adaptive Power Sampling for LLM Reasoning
  8. 2025General-Reasoner: Advancing LLM Reasoning Across All Domains