mlsanityData to 7 Oct 2026

FORTAP: Using Formulas for Numerical-Reasoning-Aware Table Pretraining

arXiv 2109.07323 ↗First posted 15 Sep 2021cs.IR cs.LGcited by 19

Not on the map. Why?

Abstract (arXiv)

Tables store rich numerical data, but numerical reasoning over tables is still a challenge. In this paper, we find that the spreadsheet formula, which performs calculations on numerical values in tables, is naturally a strong supervision of numerical reasoning. More importantly, large amounts of spreadsheets with expert-made formulae are available on the web and can be obtained easily. FORTAP is the first method for numerical-reasoning-aware table pretraining by leveraging large corpus of spreadsheet formulae. We design two formula pretraining tasks to explicitly guide FORTAP to learn numerical reference and calculation in semi-structured tables. FORTAP achieves state-of-the-art results on two representative downstream tasks, cell type classification and formula prediction, showing great potential of numerical-reasoning-aware pretraining.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 29 of its 45 references that are in the map. First posted on arXiv 15 Sep 2021.

Builds on

29 of 45 references.

References · 29

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  5. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 552
  6. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 509
  7. 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 432
  8. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 402
  9. 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 395
  10. 2021TAPEX: Table Pre-training via Learning a Neural SQL Executorcited by 233
  11. 2020GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsingcited by 171
  12. 2020Open Question Answering over Tables and Textcited by 163
17 more
  1. 2020Injecting Numerical Reasoning Skills into Language Modelscited by 133
  2. 2020Structure-Grounded Pretraining for Text-to-SQLcited by 126
  3. 2021HiTab: A Hierarchical Table Dataset for Question Answering and Natural Language Generationcited by 117
  4. 2021TABBIE: Pretrained Representations of Tabular Datacited by 111
  5. 2018Coarse-to-Fine Decoding for Neural Semantic Parsingcited by 110
  6. 2020TUTA: Tree-based Transformers for Generally Structured Table Pre-trainingcited by 100
  7. 2020Understanding tables with intermediate pre-trainingcited by 68
  8. 2021MathBERT: A Pre-Trained Model for Mathematical Formula Understandingcited by 58
  9. 2018Memory Augmented Policy Optimization for Program Synthesis and Semantic Parsingcited by 57
  10. 2019Giving BERT a Calculator: Finding Operations and Arguments with Reading Comprehensioncited by 51
  11. 2021SpreadsheetCoder: Formula Prediction from Semi-structured Contextcited by 41
  12. 2021Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skillscited by 39
  13. 2020RPT: Relational Pre-trained Transformer Is Almost All You Need towards Democratizing Data Preparationcited by 27
  14. 2021Retrieving Complex Tables with Multi-Granular Graph Representation Learningcited by 24
  15. 2021NumGPT: Improving Numeracy Ability of Generative Pre-trained Modelscited by 15
  16. 2021Learning to Reason for Text Generation from Scientific Tablescited by 13
  17. 2021TableSense: Spreadsheet Table Detection with Convolutional Neural Networkscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 6; 2024: 4; 2025: 5; 2026: 2.

19 papers.

Sort:

By area:

  1. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  2. 2023MultiTabQA: Generating Tabular Answers for Multi-Table Question Answeringcited by 32
  3. 2022TaCube: Pre-computing Data Cubes for Answering Numerical-Reasoning Questions over Tabular Datacited by 19
  4. 2024SpreadsheetLLM: Encoding Spreadsheets for Large Language Modelscited by 15
  5. 2023NameGuess: Column Name Expansion for Tabular Datacited by 13
  6. 2025Finch: Benchmarking Finance & Accounting across Spreadsheet-Centric Enterprise Workflowscited by 9
  7. 2023FormaT5: Abstention and Examples for Conditional Table Formatting with Natural Languagecited by 7
  8. 2024Table Question Answering for Low-resourced Indic Languagescited by 7
  9. 2025General Table Question Answering via Answer-Formula Joint Generationcited by 7
  10. 2025DocHop-QA: Towards Multi-Hop Reasoning over Multimodal Document Collectionscited by 7
  11. 2023Bridge the Gap between Language models and Tabular Understandingcited by 5
  12. 2022NAPG: Non-Autoregressive Program Generation for Hybrid Tabular-Textual Question Answeringcited by 4
7 more
  1. 2023QTSumm: Query-Focused Summarization over Tabular Datacited by 4
  2. 2025Formula-R1: Incentivizing LLM Reasoning over Complex Tables with Numerical Computation via Formula-Driven Reinforcement Learningcited by 3
  3. 2024Training Table Question Answering via SQL Query Decompositioncited by 1
  4. 2025FACTS: Table Summarization via Offline Template Generation with Agentic Workflowscited by 1
  5. 2023Comprehensive Solution Program Centric Pretraining for Table-and-Text Hybrid Numerical Reasoningcited by 0
  6. 2026FormulaSPIN: Self-Play Fine-Tuning for Natural Language to Spreadsheet Formula Generationcited by 0
  7. 2026Structured Prediction for Scalable Spreadsheet Table Understanding: From Cell Types to Table Ranges (Extended Version)cited by 0