RSAT: Structured Attribution Makes Small Language Models Faithful Table Reasoners
On the map: LLM reasoning → Test-time compute and efficient thinking
Abstract (arXiv)
When a language model answers a table question, users have no way to verify which cells informed which reasoning steps. We introduce RSAT, a method that trains small language models (SLMs, 1-8B) to produce step-by-step reasoning with cell-level citations grounded in table evidence. Phase 1 (SFT) teaches a structured JSON output format from verified reasoning traces. Phase 2 (GRPO) optimizes a composite reward centered on NLI-based faithfulness, alongside citation validity and parsimony. Across six models from two families-Qwen 2.5 (1.5B/3B/7B) and Llama 3 (1B/3B/8B)-RSAT improves faithfulness 3.7× over SFT alone (0.224rightarrow0.826), with near-perfect citation validity (0.992). Post-hoc attribution collapses below 13% format success, confirming that attribution must be integrated into reasoning, not retrofitted. Ablations show the faithfulness reward is essential: removing it drops faithfulness from 0.97 to 0.03.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 21 of its 22 references that are in the map. First posted on arXiv 30 Apr 2026.
Builds on
21 of 22 references.
References · 21
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
- 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
- 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
- 2023Enabling Large Language Models to Generate Text with Citationscited by 584
- 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 509
- 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 432
- 2019TabFact: A Large-scale Dataset for Table-based Fact Verificationcited by 425
- 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 395
9 more
- 2021Measuring Attribution in Natural Language Generation Modelscited by 253
- 2021TAPEX: Table Pre-training via Learning a Neural SQL Executorcited by 233
- 2022Binding Language Models in Symbolic Languagescited by 224
- 2025Reinforcement Learning for Reasoning in Small LLMs: What Works and What Doesn'tcited by 72
- 2020Understanding tables with intermediate pre-trainingcited by 68
- 2022PASTA: Table-Operations Aware Fact Verification via Sentence-Table Cloze Pre-trainingcited by 41
- 2020Program Enhanced Fact Verification with Verbalization and Graph Attention Networkcited by 29
- 2024Correctness is not Faithfulness in RAG Attributionscited by 18
- 2020Joint Verification and Reranking for Open Fact Checking Over Tablescited by 16
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025FRIT: Using Causal Importance to Improve Chain-of-Thought Faithfulness
- 2026Towards Long-Horizon Interpretability: Efficient and Faithful Multi-Token Attribution for Reasoning LLMs
- 2026ViTaB-A: Evaluating Multimodal Large Language Models on Visual Table Attribution
- 2025Table-Text Alignment: Explaining Claim Verification Against Tables in Scientific Papers
- 2023SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tables
- 2025RUST-BENCH: Benchmarking LLM Reasoning on Unstructured Text within Structured Tables
- 2024Advancing Large Language Model Attribution through Self-Improving
- 2026Learning What to Learn: Stage-Specific Data Sets for SFT-then-RL in Small Language Model Reasoning