# The Mighty ToRR: A Benchmark for Table Reasoning and Robustness

arXiv: https://arxiv.org/abs/2502.19412 · HTML: https://mlsanity.com/paper/2502.19412/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Feb 2025.

## Built on by (most cited first, 10 of 10)

- [When AI Benchmarks Plateau: A Systematic Study of Benchmark Saturation](https://mlsanity.com/paper/2602.16763/) (2026)
- [Sparks of Tabular Reasoning via Text2SQL Reinforcement Learning](https://mlsanity.com/paper/2505.00016/) (2025)
- [ErrorMap and ErrorAtlas: Charting the Failure Landscape of Large Language Models](https://mlsanity.com/paper/2601.15812/) (2026)
- [STRuCT-LLM: Unifying Tabular and Graph Reasoning with Reinforcement Learning for Semantic Parsing](https://mlsanity.com/paper/2506.21575/) (2025)
- [Beyond a Single Extractor: Re-thinking HTML-to-Text Extraction for LLM Pretraining](https://mlsanity.com/paper/2602.19548/) (2026)
- [Same Content, Different Representations: A Controlled Study for Table QA](https://mlsanity.com/paper/2509.22983/) (2025)
- [Robustness as an Emergent Property of Task Performance](https://mlsanity.com/paper/2602.03344/) (2026)
- [Financial Transaction Retrieval and Contextual Evidence for Knowledge-Grounded Reasoning](https://mlsanity.com/paper/2603.15459/) (2026)
- [Instructions Shape Production of Language, not Processing](https://mlsanity.com/paper/2605.11206/) (2026)
- [The End Of Universal Lifelong Identifiers: Identity Systems For The AI Era](https://mlsanity.com/paper/2506.02027/) (2025)

## Abstract (arXiv)

Despite its real-world significance, model performance on tabular data remains underexplored, leaving uncertainty about which model to rely on and which prompt configuration to adopt. To address this gap, we create ToRR, a benchmark for Table Reasoning and Robustness, measuring model performance and robustness on table-related tasks. The benchmark includes 10 datasets that cover different types of table reasoning capabilities across varied domains. ToRR goes beyond model performance rankings, and is designed to reflect whether models can handle tabular data consistently and robustly, across a variety of common table representation formats. We present a leaderboard as well as comprehensive analyses of the results of leading models over ToRR. Our results reveal a striking pattern of brittle model behavior, where even strong models are unable to perform robustly on tabular data tasks. Although no specific table format leads to consistently better performance, we show that testing over multiple formats is crucial for reliably estimating model capabilities. Moreover, we show that the reliability boost from testing multiple prompts can be equivalent to adding more test examples. Overall, our findings show that table understanding and reasoning tasks remain a significant challenge.