mlsanityData to 7 Oct 2026

RoTaR: Efficient Row-Based Table Representation Learning via Teacher-Student Training

arXiv 2306.11696 ↗First posted 20 Jun 2023cs.LG cs.DBcited by 0

Not on the map. Why?

Abstract (arXiv)

We propose RoTaR, a row-based table representation learning method, to address the efficiency and scalability issues faced by existing table representation learning methods. The key idea of RoTaR is to generate query-agnostic row representations that could be re-used via query-specific aggregation. In addition to the row-based architecture, we introduce several techniques: cell-aware position embedding, teacher-student training paradigm, and selective backward to improve the performance of RoTaR model.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 22 references that are in the map. First posted on arXiv 20 Jun 2023.

Builds on

18 of 22 references.

References · 18

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  4. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
  5. 2017Deep Setscited by 1,394
  6. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,153
  7. 2020Efficient Transformers: A Surveycited by 784
  8. 2019TabNet: Attentive Interpretable Tabular Learningcited by 640
  9. 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 432
  10. 2019TabFact: A Large-scale Dataset for Table-based Fact Verificationcited by 425
  11. 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 395
  12. 2021TAPEX: Table Pre-training via Learning a Neural SQL Executorcited by 233
6 more
  1. 2022UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Modelscited by 231
  2. 2021TABBIE: Pretrained Representations of Tabular Datacited by 111
  3. 2019On the Limitations of Representing Functions on Setscited by 89
  4. 2020Understanding tables with intermediate pre-trainingcited by 68
  5. 2021MATE: Multi-view Attention for Table Transformer Efficiencycited by 54
  6. 2020RPT: Relational Pre-trained Transformer Is Almost All You Need towards Democratizing Data Preparationcited by 27

Built on by

Not cited yet.