mlsanityData to 7 Oct 2026

TableLoRA: Low-rank Adaptation on Table Structure Understanding for Large Language Models

arXiv 2503.04396 ↗First posted 6 Mar 2025cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

Tabular data are crucial in many fields and their understanding by large language models (LLMs) under high parameter efficiency paradigm is important. However, directly applying parameter-efficient fine-tuning (PEFT) techniques to tabular tasks presents significant challenges, particularly in terms of better table serialization and the representation of two-dimensional structured information within a one-dimensional sequence. To address this, we propose TableLoRA, a module designed to improve LLMs' understanding of table structure during PEFT. It incorporates special tokens for serializing tables with special token encoder and uses 2D LoRA to encode low-rank information on cell positions. Experiments on four tabular-related datasets demonstrate that TableLoRA consistently outperforms vanilla LoRA and surpasses various table encoding methods tested in control experiments. These findings reveal that TableLoRA, as a table-specific LoRA, enhances the ability of LLMs to process tabular data effectively, especially in low-parameter settings, demonstrating its potential as a robust solution for handling table-related tasks.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 27 of its 39 references that are in the map. First posted on arXiv 6 Mar 2025.

Builds on

27 of 39 references.

References · 27

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  3. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  4. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
  5. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  6. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 509
  7. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 442
  8. 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 432
  9. 2019TabFact: A Large-scale Dataset for Table-based Fact Verificationcited by 425
  10. 2021Learning How to Ask: Querying LMs with Mixtures of Soft Promptscited by 376
  11. 2020HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Datacited by 263
  12. 2021TAPEX: Table Pre-training via Learning a Neural SQL Executorcited by 233
15 more
  1. 2024Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understandingcited by 185
  2. 2021FeTaQA: Free-form Table Question Answeringcited by 174
  3. 2023TableLlama: Towards Open Large Generalist Models for Tablescited by 136
  4. 2021HiTab: A Hierarchical Table Dataset for Question Answering and Natural Language Generationcited by 117
  5. 2021TABBIE: Pretrained Representations of Tabular Datacited by 111
  6. 2020TUTA: Tree-based Transformers for Generally Structured Table Pre-trainingcited by 100
  7. 2024TableLLM: Enabling Tabular Data Manipulation by LLMs in Real Office Usage Scenarioscited by 76
  8. 2021MATE: Multi-view Attention for Table Transformer Efficiencycited by 54
  9. 2022Table Pre-training: A Survey on Model Architectures, Pre-training Objectives, and Downstream Taskscited by 42
  10. 2023Text2Analysis: A Benchmark of Table Question Answering with Advanced Data Analysis and Unclear Queriescited by 38
  11. 2023TAP4LLM: Table Provider on Sampling, Augmenting, and Packing Semi-structured Data for Large Language Model Reasoningcited by 32
  12. 2024StructLM: Towards Building Generalist Models for Structured Knowledge Groundingcited by 29
  13. 2020Table2Charts: Recommending Charts by Learning Shared Table Representationscited by 14
  14. 2021DoT: An efficient Double Transformer for NLP tasks with tablescited by 8
  15. 2022AnaMeta: A Table Understanding Dataset of Field Metadata Knowledge Shared by Multi-dimensional Data Analysis Taskscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025Table-r1: Self-supervised and Reinforcement Learning for Program-based Table Reasoning in Small Language Modelscited by 7
  2. 2025Table Question Answering in the Era of Large Language Models: A Comprehensive Survey of Tasks, Methods, and Evaluationcited by 4
  3. 2025Prompt Orchestration Markup Languagecited by 1
  4. 2026How Do Language Models Understand Tables? A Mechanistic Analysis of Cell Locationcited by 0