# Analyzing LLM Instruction Optimization for Tabular Fact Verification

arXiv: https://arxiv.org/abs/2602.17937 · HTML: https://mlsanity.com/paper/2602.17937/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 27 of its 42 references that are in the map.
- First posted on arXiv 20 Feb 2026.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)

## Builds on (most cited first, 27 of 27)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [TAPAS: Weakly Supervised Table Parsing via Pre-training](https://mlsanity.com/paper/2004.02349/) (2020)
- [TabFact: A Large-scale Dataset for Table-based Fact Verification](https://mlsanity.com/paper/1909.02164/) (2019)
- [Do Prompt-Based Models Really Understand the Meaning of their Prompts?](https://mlsanity.com/paper/2109.01247/) (2021)
- [Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs](https://mlsanity.com/paper/2406.11695/) (2024)
- [TAPEX: Table Pre-training via Learning a Neural SQL Executor](https://mlsanity.com/paper/2107.07653/) (2021)
- [Binding Language Models in Symbolic Languages](https://mlsanity.com/paper/2210.02875/) (2022)
- [AVeriTeC: A Dataset for Real-world Claim Verification with Evidence from the Web](https://mlsanity.com/paper/2305.13117/) (2023)
- [TableLlama: Towards Open Large Generalist Models for Tables](https://mlsanity.com/paper/2311.09206/) (2023)
- [ReAcTable: Enhancing ReAct for Table Question Answering](https://mlsanity.com/paper/2310.00815/) (2023)
- [Understanding tables with intermediate pre-training](https://mlsanity.com/paper/2010.00571/) (2020)
- [SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tables](https://mlsanity.com/paper/2305.13186/) (2023)
- [The language of prompting: What linguistic properties make a prompt successful?](https://mlsanity.com/paper/2311.01967/) (2023)
- [SemEval-2021 Task 9: Fact Verification and Evidence Finding for Tabular Data in Scientific Documents (SEM-TAB-FACTS)](https://mlsanity.com/paper/2105.13995/) (2021)
- [Program Enhanced Fact Verification with Verbalization and Graph Attention Network](https://mlsanity.com/paper/2010.03084/) (2020)
- [LogicalFactChecker: Leveraging Logical Operations for Fact Checking with Graph Module Network](https://mlsanity.com/paper/2004.13659/) (2020)
- [BlendSQL: A Scalable Dialect for Unifying Hybrid Question Answering in Relational Algebra](https://mlsanity.com/paper/2402.17882/) (2024)
- [ProTrix: Building Models for Planning and Reasoning over Tables with Sentence Context](https://mlsanity.com/paper/2403.02177/) (2024)
- [H-STAR: LLM-driven Hybrid SQL-Text Adaptive Reasoning on Tables](https://mlsanity.com/paper/2407.05952/) (2024)
- [TableMind: An Autonomous Programmatic Agent for Tool-Augmented Table Reasoning](https://mlsanity.com/paper/2509.06278/) (2025)
- [SciVer: Evaluating Foundation Models for Multimodal Scientific Claim Verification](https://mlsanity.com/paper/2506.15569/) (2025)
- [Does Table Source Matter? Benchmarking and Improving Multimodal Scientific Table Understanding and Reasoning](https://mlsanity.com/paper/2501.13042/) (2025)
- [Efficient Multi-Agent Collaboration with Tool Use for Online Planning in Complex Table Question Answering](https://mlsanity.com/paper/2412.20145/) (2024)
- [Exploring Decomposition for Table-based Fact Verification](https://mlsanity.com/paper/2109.11020/) (2021)
- [TART: An Open-Source Tool-Augmented Framework for Explainable Table-based Reasoning](https://mlsanity.com/paper/2409.11724/) (2024)
- [Exploring the Robustness of Language Models for Tabular Question Answering via Attention Analysis](https://mlsanity.com/paper/2406.12719/) (2024)
- [Atomic Reasoning for Scientific Table Claim Verification](https://mlsanity.com/paper/2506.06972/) (2025)

## Built on by (most cited first, 1 of 1)

- [Benchmarking Prompt Optimization of Large Language Models With Chess](https://mlsanity.com/paper/2610.00416/) (2026)

## Abstract (arXiv)

Instruction optimization provides a lightweight, model-agnostic approach to enhancing the reasoning performance of large language models (LLMs). This paper presents the first systematic comparison of instruction optimization, based on the DSPy optimization framework, for tabular fact verification. We evaluate four out-of-the-box prompting techniques that cover both text-only prompting and code use: direct prediction, Chain-of-Thought (CoT), ReAct with SQL tools, and CodeAct with Python execution. We study three optimizers from the DSPy framework -- COPRO, MiPROv2, and SIMBA -- across four benchmarks and three model families. We find that instruction optimization consistently improves verification accuracy, with MiPROv2 yielding the most stable gains for CoT, and SIMBA providing the largest benefits for ReAct agents, particularly at larger model scales. Behavioral analyses reveal that SIMBA encourages more direct reasoning paths by applying heuristics, thereby improving numerical comparison abilities in CoT reasoning and helping avoid unnecessary tool calls in ReAct agents. Across different prompting techniques, CoT remains effective for tabular fact checking, especially with smaller models. Although ReAct agents built with larger models can achieve competitive performance, they require careful instruction optimization.