TaTToo: Tool-Grounded Thinking PRM for Test-Time Scaling in Tabular Reasoning
On the map: LLM reasoning → Reinforcement learning for reasoning
Abstract (arXiv)
Process Reward Models (PRMs) have recently emerged as a powerful framework for enhancing the reasoning capabilities of large reasoning models (LRMs), particularly in the context of test-time scaling (TTS). However, their potential for supervising LRMs on tabular reasoning domains remains underexplored. Through detailed empirical analyses, we identify that existing PRMs, though widely adopted for supervising text-only reasoning steps, struggle with table-specific operations such as sub-table retrieval and schema interaction, leading to critical performance bottlenecks. To address this limitation, we propose TaTToo, a novel table-grounded PRM framework that (i) reasons explicitly over tabular reasoning steps and (ii) integrates tool-based verification to provide precise reward supervision. Concretely, we first design a scalable data curation pipeline that constructs over 60k high-quality step-level annotations by integrating table verification rationales with tool-based executions. Building on the collected data, we train TaTToo with a dual-stage paradigm: cold-start supervised fine-tuning to capture tool-use reasoning patterns, followed by reinforcement learning with tool-grounded reward shaping to align our model with table-based verification. We provide a comprehensive evaluation of the policy improvement induced by our newly designed PRM. Across 5 challenging tabular reasoning benchmarks covering numerical reasoning, fact-checking, and data analysis, TaTToo improves downstream policy LRMs by 30.9% at inference, surpasses strong PRM baselines such as Qwen-2.5-Math-PRM-72B with only 8B parameters, and demonstrates strong generalizability across diverse TTS strategies.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 60 of its 85 references that are in the map. First posted on arXiv 7 Oct 2025.
Builds on
60 of 85 references.
Closest ancestors · 15
Same direction, newest first
- 2025Agentic Reinforced Policy Optimization26 Jul
- 2025ReasonFlux-PRM: Trajectory-Aware PRMs for Long Chain-of-Thought Reasoning in LLMs23 Jun
- 2025Is PRM Necessary? Problem-Solving RL Implicitly Induces PRM Capability in LLMs16 May
- 2025RM-R1: Reward Modeling as Reasoning5 May
- 2025Process Reward Models That Think23 Apr
- 2025GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoning1 Apr
- 2025Process Reinforcement through Implicit Rewards3 Feb
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
- 2025The Lessons of Developing Process Reward Models in Mathematical Reasoning13 Jan
- 2024Process Reward Model with Q-Value Rankings15 Oct
- 2024Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning10 Oct
- 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search6 Jun
3 more
Other references · 45
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2025Qwen3 Technical Reportcited by 8,195
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
- 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
- 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
- 2025s1: Simple test-time scalingcited by 1,311
- 2024OpenAI o1 System Cardcited by 1,220
- 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
- 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
- 2025LIMO: Less is More for Reasoningcited by 488
- 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 432
33 more
- 2019TabFact: A Large-scale Dataset for Table-based Fact Verificationcited by 425
- 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 395
- 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMscited by 377
- 2025ToolRL: Reward is All Tool Learning Needscited by 342
- 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 308
- 2020HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Datacited by 263
- 2023ART: Automatic multi-step reasoning and tool-use for large language modelscited by 193
- 2024Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understandingcited by 185
- 2023TableLlama: Towards Open Large Generalist Models for Tablescited by 136
- 2025Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scalingcited by 135
- 2023RestGPT: Connecting Large Language Models with Real-World RESTful APIscited by 129
- 2024TableBench: A Comprehensive and Complex Benchmark for Table Question Answeringcited by 127
- 2021TABBIE: Pretrained Representations of Tabular Datacited by 111
- 2024OpenR: An Open Source Framework for Advanced Reasoning with Large Language Modelscited by 92
- 2023Table-GPT: Table-tuned GPT for Diverse Table Taskscited by 69
- 2024TableGPT2: A Large Multimodal Model with Tabular Data Integrationcited by 69
- 2025OctoTools: An Agentic Framework with Extensible Tools for Complex Reasoningcited by 69
- 2024LLM With Tools: A Surveycited by 60
- 2025A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Futurecited by 53
- 2025Table-R1: Region-based Reinforcement Learning for Table Understandingcited by 35
- 2023Exploring the Numerical Reasoning Capabilities of Language Models: A Comprehensive Analysis on Tabular Datacited by 33
- 2023TAP4LLM: Table Provider on Sampling, Augmenting, and Packing Semi-structured Data for Large Language Model Reasoningcited by 32
- 2024A Survey of Table Reasoning with Large Language Modelscited by 29
- 2020RPT: Relational Pre-trained Transformer Is Almost All You Need towards Democratizing Data Preparationcited by 27
- 2025Table-R1: Inference-Time Scaling for Table Reasoningcited by 23
- 2024When LLMs Meet Cunning Texts: A Fallacy Understanding Benchmark for Large Language Modelscited by 22
- 2023Towards Real-World Writing Assistance: A Chinese Character Checking Benchmark with Faked and Misspelled Characterscited by 21
- 2025Reasoning-Table: Exploring Reinforcement Learning for Table Reasoningcited by 18
- 2025RAG over Tables: Hierarchical Memory Index, Multi-Stage Retrieval, and Benchmarkingcited by 17
- 2024From Natural Language to SQL: Review of LLM-based Text-to-SQL Systemscited by 15
- 2017TableQA: Question Answering on Tabular Datacited by 8
- 2025MT-RAIG: Novel Benchmark and Evaluation Framework for Retrieval-Augmented Insight Generation over Multiple Tablescited by 7
- 2025Table-r1: Self-supervised and Reinforcement Learning for Program-based Table Reasoning in Small Language Modelscited by 7
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
- 2026DIVERSED: Relaxed Speculative Decoding via Dynamic Ensemble Verificationcited by 5
- 2026Rewarding the Scientific Process: Process-Level Reward Modeling for Agentic Data Analysiscited by 4
- 2026A Table Is Worth 64 Tokens: Pixel-level Compression for Multi-Table Document Question Answeringcited by 0
- 2026Towards Reliable AI Data Scientists: Data Agents with Workflow Harnessescited by 0
- 2026Towards Reliable AI Data Scientists: Data Agents with Workflow Harnesses28 Sep 2026
- 2026A Table Is Worth 64 Tokens: Pixel-level Compression for Multi-Table Document Question Answering27 Aug 2026
- 2026Rewarding the Scientific Process: Process-Level Reward Modeling for Agentic Data Analysis27 Apr 2026
- 2026DIVERSED: Relaxed Speculative Decoding via Dynamic Ensemble Verification8 Apr 2026
- 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agents18 Jan 2026
Similar, not cited
Close in wording, no citation either way
- 2025Athena: Enhancing Multimodal Reasoning with Data-efficient Process Reward Models
- 2025A Survey of Process Reward Models: From Outcome Signals to Process Supervisions for Large Language Models
- 2025From Mathematical Reasoning to Code: Generalization of Process Reward Models in Test-Time Scaling
- 2026SCI-PRM: A Tool Aware Process Reward Model for Scientific Reasoning Verification
- 2025PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Models
- 2025GroundedPRM: Tree-Guided and Fidelity-Aware Process Reward Modeling for Step-Level Reasoning
- 2023Let's reward step by step: Step-Level reward model as the Navigators for Reasoning
- 2026Enhancing Table Reasoning with Deterministic Table-State Rewards