mlsanityData to 14 Oct 2026

TUTA: Tree-based Transformers for Generally Structured Table Pre-training

arXiv 2010.12537 ↗First posted 21 Oct 2020cs.IR cs.AI cs.DBcited by 100

Not on the map. Why?

Abstract (arXiv)

Tables are widely used with various structures to organize and present data. Recent attempts on table understanding mainly focus on relational tables, yet overlook to other common table structures. In this paper, we propose TUTA, a unified pre-training architecture for understanding generally structured tables. Noticing that understanding a table requires spatial, hierarchical, and semantic information, we enhance transformers with three novel structure-aware mechanisms. First, we devise a unified tree-based structure, called a bi-dimensional coordinate tree, to describe both the spatial and hierarchical information of generally structured tables. Upon this, we propose tree-based attention and position embedding to better capture the spatial and hierarchical information. Moreover, we devise three progressive pre-training objectives to enable representations at the token, cell, and table levels. We pre-train TUTA on a wide range of unlabeled web and spreadsheet tables and fine-tune it on two critical tasks in the field of table structure understanding: cell type classification and table type classification. Experiments show that TUTA is highly effective, achieving state-of-the-art on five widely-studied datasets.

Key facts. Cited by 100 papers in the map (data through 14 Oct 2026). First posted on arXiv 21 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 22; 2023: 19; 2024: 21; 2025: 23; 2026: 11.

100 papers.

Sort:

By area:

  1. 2021TAPEX: Table Pre-training via Learning a Neural SQL Executorcited by 235
  2. 2022UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Modelscited by 231
  3. 2024Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understandingcited by 188
  4. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 159
  5. 2023TableLlama: Towards Open Large Generalist Models for Tablescited by 137
  6. 2022MultiHiertt: Numerical Reasoning over Multi Hierarchical Tabular and Textual Datacited by 121
  7. 2023Table Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Studycited by 116
  8. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  9. 2022TableFormer: Robust Transformer Modeling for Table-Text Encodingcited by 91
  10. 2022Towards Complex Document Understanding By Discrete Reasoningcited by 82
  11. 2023Generative Table Pre-training Empowers Models for Tabular Predictioncited by 55
  12. 2023Tabular Representation, Noisy Operators, and Impacts on Table Structure Understanding Tasks in LLMscited by 49
38 more
  1. 2022Table Pre-training: A Survey on Model Architectures, Pre-training Objectives, and Downstream Taskscited by 42
  2. 2021Annotating Columns with Pre-trained Language Modelscited by 37
  3. 2023HYTREL: Hypergraph-enhanced Tabular Data Representation Learningcited by 36
  4. 2023TAP4LLM: Table Provider on Sampling, Augmenting, and Packing Semi-structured Data for Large Language Model Reasoningcited by 32
  5. 2022PTab: Using the Pre-trained Language Model for Modeling Tabular Datacited by 30
  6. 2022OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answeringcited by 29
  7. 2022ReasTAP: Injecting Table Reasoning Skills During Pre-training via Synthetic Reasoning Examplescited by 29
  8. 2023UniTabE: A Universal Pretraining Protocol for Tabular Foundation Model in Data Sciencecited by 29
  9. 2025Large Language Models are Good Relational Learnerscited by 24
  10. 2024ProTrix: Building Models for Planning and Reasoning over Tables with Sentence Contextcited by 23
  11. 2022Table-To-Text generation and pre-training with TabT5cited by 20
  12. 2021FORTAP: Using Formulas for Numerical-Reasoning-Aware Table Pretrainingcited by 19
  13. 2024Language Modeling on Tabular Data: A Survey of Foundations, Techniques and Evolutioncited by 19
  14. 2022Tele-Knowledge Pre-training for Fault Analysiscited by 18
  15. 2025TableMaster: A Recipe to Advance Table Understanding with Language Modelscited by 18
  16. 2024SpreadsheetLLM: Encoding Spreadsheets for Large Language Modelscited by 17
  17. 2023Structure Pretraining and Prompt Tuning for Knowledge Graph Transfercited by 16
  18. 2025LLM/Agent-as-Data-Analyst: A Surveycited by 16
  19. 2022DeepJoin: Joinable Table Discovery with Pre-trained Language Modelscited by 15
  20. 2023Auto-Tables: Synthesizing Multi-Step Transformations to Relationalize Tables without Using Examplescited by 15
  21. 2025Multimodal Tabular Reasoning with Privileged Structured Informationcited by 15
  22. 2023API-Assisted Code Generation for Question Answering on Varied Table Structurescited by 14
  23. 2024PoTable: Towards Systematic Thinking via Plan-then-Execute Stage Reasoning on Tablescited by 14
  24. 2023Observatory: Characterizing Embeddings of Relational Tablescited by 12
  25. 2024Unlock the Potential of Large Language Models for Predictive Tabular Tasks in Data Science with Table-Specific Pretrainingcited by 12
  26. 2022Table Retrieval May Not Necessitate Table-specific Model Designcited by 11
  27. 2023LakeBench: Benchmarks for Data Discovery over Data Lakescited by 10
  28. 2023A Hierarchical Spatial Transformer for Massive Point Samples in Continuous Spacecited by 10
  29. 2024Accurate and Regret-aware Numerical Problem Solver for Tabular Question Answeringcited by 10
  30. 2025Tabular Data Understanding with LLMs: A Survey of Recent Advances and Challengescited by 10
  31. 2025Finch: Benchmarking Finance & Accounting across Spreadsheet-Centric Enterprise Workflowscited by 10
  32. 2024NL2Formula: Generating Spreadsheet Formulas from Natural Language Queriescited by 8
  33. 2025In-context Learning of Evolving Data Streams with Tabular Foundational Modelscited by 8
  34. 2023FormaT5: Abstention and Examples for Conditional Table Formatting with Natural Languagecited by 7
  35. 2025Hybrid Graphs for Table-and-Text based Question Answering using LLMscited by 7
  36. 2022Table Structure Recognition with Conditional Attentioncited by 6
  37. 2022Optimization Techniques for Unsupervised Complex Table Reasoning via Self-Training Frameworkcited by 6
  38. 2023PixT3: Pixel-based Table-To-Text Generationcited by 6

50 most cited of 100 All 100 (JSON)