mlsanityData to 14 Oct 2026

Test-Time Scaling with Reflective Generative Model

arXiv 2507.01951 ↗First posted 2 Jul 2025cs.LG cs.CLcited by 4

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

We introduce our first reflective generative model MetaStone-S1, which obtains OpenAI o3-mini's performance via the new Reflective Generative Form. The new form focuses on high-quality reasoning trajectory selection and contains two novelties: 1) A unified interface for policy and process reward model: we share the backbone network and use task-specific heads for reasoning trajectory predicting and scoring respectively, introducing only 53M extra parameters for trajectory scoring. 2) Eliminating the reliance on process-level annotation: we provide a self-supervised process reward model, which can directly learn the high-quality reasoning trajectory selection from the outcome reward. Equipped with the reflective generative form, MetaStone-S1 is naturally suitable for test-time scaling, and we provide three reasoning effort modes (low, medium, and high) based on the controllable thinking length. Experiments demonstrate that our MetaStone-S1 achieves comparable performance to OpenAI o3-mini's series with only 32B parameter size. To support the research community, we have open-sourced MetaStone-S1 at https://github.com/MetaStone-AI/MetaStone-S1.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 23 of its 30 references that are in the map. First posted on arXiv 2 Jul 2025.

Builds on

23 of 30 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Revisiting the Test-Time Scaling of o1-like Models: Do they Truly Possess Test-Time Scaling Capabilities?17 Feb
  2. 2025Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling10 Feb
  3. 2025Demystifying Long Chain-of-Thought Reasoning in LLMs5 Feb
  4. 2025s1: Simple test-time scaling31 Jan
  5. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs30 Dec
  6. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug

Other references · 17

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  3. 2023Let's Verify Step by Stepcited by 3,789
  4. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  5. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  6. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  7. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
  8. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 708
  9. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 441
  10. 2025The Lessons of Developing Process Reward Models in Mathematical Reasoningcited by 415
  11. 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 344
  12. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 313
5 more
  1. 2024The Impact of Reasoning Step Length on Large Language Modelscited by 153
  2. 2025LIMR: Less is More for RL Scalingcited by 139
  3. 2024Unlocking the Capabilities of Thought: A Reasoning Boundary Framework to Quantify and Optimize Chain-of-Thoughtcited by 56
  4. 2025Exploring the Limit of Outcome Reward for Learning Mathematical Reasoningcited by 52
  5. 2025AURORA:Automated Training Framework of Universal Process Reward Models via Ensemble Prompting and Reverse Verificationcited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025A Survey of Process Reward Models: From Outcome Signals to Process Supervisions for Large Language Modelscited by 38
  2. 2025Best-of-∞ -- Asymptotic Performance of Test-Time LLM Ensemblingcited by 3
  3. 2025TrajSelector: Harnessing Latent Representations for Efficient and Effective Best-of-N in Large Reasoning Modelcited by 3
  4. 2026Towards Robust Process Reward Modeling via Noise-aware Learningcited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoning
  2. 2025StepWiser: Stepwise Generative Judges for Wiser Reasoning
  3. 2025ReProbe: Efficient Test-Time Scaling of Multi-Step Reasoning by Probing Internal States of Large Language Models
  4. 2025Solve-Detect-Verify: Inference-Time Scaling with Flexible Generative Verifier
  5. 2026CoT2-Meta: Budgeted Metacognitive Control for Test-Time Reasoning
  6. 2026ReflectRL: Learning from Golden Negative Trajectories via Reflective-to-Direct Reasoning
  7. 2025Extending Test-Time Scaling: A 3D Perspective with Context, Batch, and Turn
  8. 2025ReasonFlux-PRM: Trajectory-Aware PRMs for Long Chain-of-Thought Reasoning in LLMs