# MultiHiertt: Numerical Reasoning over Multi Hierarchical Tabular and Textual Data

arXiv: https://arxiv.org/abs/2206.01347 · HTML: https://mlsanity.com/paper/2206.01347/

## Key facts

- Cited by 121 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 66 references that are in the map.
- First posted on arXiv 3 Jun 2022.

## Builds on (most cited first, 30 of 47)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs](https://mlsanity.com/paper/1903.00161/) (2019)
- [Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning](https://mlsanity.com/paper/1709.00103/) (2017)
- [MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms](https://mlsanity.com/paper/1905.13319/) (2019)
- [FinQA: A Dataset of Numerical Reasoning over Financial Data](https://mlsanity.com/paper/2109.00122/) (2021)
- [Compositional Semantic Parsing on Semi-Structured Tables](https://mlsanity.com/paper/1508.00305/) (2015)
- [TAPAS: Weakly Supervised Table Parsing via Pre-training](https://mlsanity.com/paper/2004.02349/) (2020)
- [The Web as a Knowledge-base for Answering Complex Questions](https://mlsanity.com/paper/1803.06643/) (2018)
- [TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance](https://mlsanity.com/paper/2105.07624/) (2021)
- [FinBERT: Financial Sentiment Analysis with Pre-trained Language Models](https://mlsanity.com/paper/1908.10063/) (2019)
- [Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases](https://mlsanity.com/paper/1909.03683/) (2019)
- [HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Data](https://mlsanity.com/paper/2004.07347/) (2020)
- [UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models](https://mlsanity.com/paper/2201.05966/) (2022)
- [MultiModalQA: Complex Question Answering over Text, Tables and Images](https://mlsanity.com/paper/2104.06039/) (2021)
- [FeTaQA: Free-form Table Question Answering](https://mlsanity.com/paper/2104.00369/) (2021)
- [FinBERT: A Pretrained Language Model for Financial Communications](https://mlsanity.com/paper/2006.08097/) (2020)
- [Global Table Extractor (GTE): A Framework for Joint Table Identification and Cell Structure Recognition Using Visual Context](https://mlsanity.com/paper/2005.00589/) (2020)
- [Injecting Numerical Reasoning Skills into Language Models](https://mlsanity.com/paper/2004.04487/) (2020)
- [Unsupervised Question Decomposition for Question Answering](https://mlsanity.com/paper/2002.09758/) (2020)
- [HiTab: A Hierarchical Table Dataset for Question Answering and Natural Language Generation](https://mlsanity.com/paper/2108.06712/) (2021)
- [Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-trained Language Models](https://mlsanity.com/paper/2005.00683/) (2020)

## Built on by (most cited first, 30 of 121)

- [Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models](https://mlsanity.com/paper/2503.06749/) (2025)
- [Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning](https://mlsanity.com/paper/2209.14610/) (2022)
- [What matters when building vision-language models?](https://mlsanity.com/paper/2405.02246/) (2024)
- [Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Search](https://mlsanity.com/paper/2412.18319/) (2024)
- [Building and better understanding vision-language models: insights and future directions](https://mlsanity.com/paper/2408.12637/) (2024)
- [A Survey of Deep Learning for Mathematical Reasoning](https://mlsanity.com/paper/2212.10535/) (2022)
- [Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective](https://mlsanity.com/paper/2506.14965/) (2025)
- [Eagle 2: Building Post-Training Data Strategies from Scratch for Frontier Vision-Language Models](https://mlsanity.com/paper/2501.14818/) (2025)
- [Are LLMs Capable of Data-based Statistical and Causal Reasoning? Benchmarking Advanced Quantitative Reasoning with Data](https://mlsanity.com/paper/2402.17644/) (2024)
- [EHRXQA: A Multi-Modal Question Answering Dataset for Electronic Health Records with Chest X-ray Images](https://mlsanity.com/paper/2310.18652/) (2023)
- [FineVision: Open Data Is All You Need](https://mlsanity.com/paper/2510.17269/) (2025)
- [DeepAnalyze: Agentic Large Language Models for Autonomous Data Science](https://mlsanity.com/paper/2510.16872/) (2025)
- [DocMath-Eval: Evaluating Math Reasoning Capabilities of LLMs in Understanding Long and Specialized Documents](https://mlsanity.com/paper/2311.09805/) (2023)
- [ReasTAP: Injecting Table Reasoning Skills During Pre-training via Synthetic Reasoning Examples](https://mlsanity.com/paper/2210.12374/) (2022)
- [StructLM: Towards Building Generalist Models for Structured Knowledge Grounding](https://mlsanity.com/paper/2402.16671/) (2024)
- [LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding](https://mlsanity.com/paper/2605.27365/) (2026)
- [Is It Really Long Context if All You Need Is Retrieval? Towards Genuinely Difficult Long Context NLP](https://mlsanity.com/paper/2407.00402/) (2024)
- [Evaluating LLMs' Mathematical Reasoning in Financial Document Question Answering](https://mlsanity.com/paper/2402.11194/) (2024)
- [Mathematical Language Models: A Survey](https://mlsanity.com/paper/2312.07622/) (2023)
- [TAT-LLM: A Specialized Language Model for Discrete Reasoning over Tabular and Textual Data](https://mlsanity.com/paper/2401.13223/) (2024)
- [Language Modeling on Tabular Data: A Survey of Foundations, Techniques and Evolution](https://mlsanity.com/paper/2408.10548/) (2024)
- [FinDVer: Explainable Claim Verification over Long and Hybrid-Content Financial Documents](https://mlsanity.com/paper/2411.05764/) (2024)
- [MMHQA-ICL: Multimodal In-context Learning for Hybrid Question Answering over Text, Tables and Images](https://mlsanity.com/paper/2309.04790/) (2023)
- [Leopard: A Vision Language Model For Text-Rich Multi-Image Tasks](https://mlsanity.com/paper/2410.01744/) (2024)
- [RAG over Tables: Hierarchical Memory Index, Multi-Stage Retrieval, and Benchmarking](https://mlsanity.com/paper/2504.01346/) (2025)
- [Answering Numerical Reasoning Questions in Table-Text Hybrid Contents with Graph-based Encoder and Tree-based Decoder](https://mlsanity.com/paper/2209.07692/) (2022)
- [S3HQA: A Three-Stage Approach for Multi-hop Text-Table Hybrid Question Answering](https://mlsanity.com/paper/2305.11725/) (2023)
- [SEC-QA: A Systematic Evaluation Corpus for Financial QA](https://mlsanity.com/paper/2406.14394/) (2024)
- [TQA-Bench: Evaluating LLMs for Multi-Table Question Answering](https://mlsanity.com/paper/2411.19504/) (2024)
- [FinDER: Financial Dataset for Question Answering and Evaluating Retrieval-Augmented Generation](https://mlsanity.com/paper/2504.15800/) (2025)

## Abstract (arXiv)

Numerical reasoning over hybrid data containing both textual and tabular content (e.g., financial reports) has recently attracted much attention in the NLP community. However, existing question answering (QA) benchmarks over hybrid data only include a single flat table in each document and thus lack examples of multi-step numerical reasoning across multiple hierarchical tables. To facilitate data analytical progress, we construct a new large-scale benchmark, MultiHiertt, with QA pairs over Multi Hierarchical Tabular and Textual data. MultiHiertt is built from a wealth of financial reports and has the following unique characteristics: 1) each document contain multiple tables and longer unstructured texts; 2) most of tables contained are hierarchical; 3) the reasoning process required for each question is more complex and challenging than existing benchmarks; and 4) fine-grained annotations of reasoning processes and supporting facts are provided to reveal complex numerical reasoning. We further introduce a novel QA model termed MT2Net, which first applies facts retrieving to extract relevant supporting facts from both tables and text and then uses a reasoning module to perform symbolic reasoning over retrieved facts. We conduct comprehensive experiments on various baselines. The experimental results show that MultiHiertt presents a strong challenge for existing baselines whose results lag far behind the performance of human experts. The dataset and code are publicly available at https://github.com/psunlpgroup/MultiHiertt.