# TABLET: Learning From Instructions For Tabular Data

arXiv: https://arxiv.org/abs/2304.13188 · HTML: https://mlsanity.com/paper/2304.13188/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 41 references that are in the map.
- First posted on arXiv 25 Apr 2023.

## Builds on (most cited first, 21 of 21)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://mlsanity.com/paper/2110.08207/) (2021)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [Cross-Task Generalization via Natural Language Crowdsourcing Instructions](https://mlsanity.com/paper/2104.08773/) (2021)
- [Tabular Data: Deep Learning is Not All You Need](https://mlsanity.com/paper/2106.03253/) (2021)
- [MetaICL: Learning to Learn In Context](https://mlsanity.com/paper/2110.15943/) (2021)
- [TabTransformer: Tabular Data Modeling Using Contextual Embeddings](https://mlsanity.com/paper/2012.06678/) (2020)
- [One Embedder, Any Task: Instruction-Finetuned Text Embeddings](https://mlsanity.com/paper/2212.09741/) (2022)
- [Language Models are Realistic Tabular Data Generators](https://mlsanity.com/paper/2210.06280/) (2022)
- [TabLLM: Few-shot Classification of Tabular Data with Large Language Models](https://mlsanity.com/paper/2210.10723/) (2022)
- [A Semantic Loss Function for Deep Learning with Symbolic Knowledge](https://mlsanity.com/paper/1711.11157/) (2017)
- [Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systems](https://mlsanity.com/paper/1903.12394/) (2019)
- [Can Foundation Models Wrangle Your Data?](https://mlsanity.com/paper/2205.09911/) (2022)
- [LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Tasks](https://mlsanity.com/paper/2206.06565/) (2022)
- [Harnessing Deep Neural Networks with Logic Rules](https://mlsanity.com/paper/1603.06318/) (2016)
- [Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections](https://mlsanity.com/paper/2104.04670/) (2021)
- [DDXPlus: A New Dataset For Automatic Medical Diagnosis](https://mlsanity.com/paper/2205.09148/) (2022)
- [Guess the Instruction! Flipped Learning Makes Language Models Stronger Zero-Shot Learners](https://mlsanity.com/paper/2210.02969/) (2022)

## Built on by (most cited first, 28 of 28)

- [Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs](https://mlsanity.com/paper/2307.03393/) (2023)
- [Large Language Models Can Automatically Engineer Features for Few-Shot Tabular Learning](https://mlsanity.com/paper/2404.09491/) (2024)
- [Evaluating language models as risk scores](https://mlsanity.com/paper/2407.14614/) (2024)
- [Emergency Department Decision Support using Clinical Pseudo-notes](https://mlsanity.com/paper/2402.00160/) (2024)
- [Language Modeling on Tabular Data: A Survey of Foundations, Techniques and Evolution](https://mlsanity.com/paper/2408.10548/) (2024)
- [Confronting LLMs with Traditional ML: Rethinking the Fairness of Large Language Models in Tabular Classifications](https://mlsanity.com/paper/2310.14607/) (2023)
- [Unlock the Potential of Large Language Models for Predictive Tabular Tasks in Data Science with Table-Specific Pretraining](https://mlsanity.com/paper/2403.20208/) (2024)
- [DP-TabICL: In-Context Learning with Differentially Private Tabular Data](https://mlsanity.com/paper/2403.05681/) (2024)
- [Improving LLM Group Fairness on Tabular Data via In-Context Learning](https://mlsanity.com/paper/2412.04642/) (2024)
- [Emergent Bias and Fairness in Multi-Agent Decision Systems](https://mlsanity.com/paper/2512.16433/) (2025)
- [FORESTLLM: Large Language Models Make Random Forest Great on Few-shot Tabular Learning](https://mlsanity.com/paper/2601.11311/) (2026)
- [Transfer Learning of Tabular Data by Finetuning Large Language Models](https://mlsanity.com/paper/2501.06863/) (2025)
- [Transformers Boost the Performance of Decision Trees on Tabular Data across Sample Sizes](https://mlsanity.com/paper/2502.02672/) (2025)
- [Latte: Transfering LLMs` Latent-level Knowledge for Few-shot Tabular Learning](https://mlsanity.com/paper/2505.05237/) (2025)
- [Reasoning Factual Knowledge in Structured Data with Large Language Models](https://mlsanity.com/paper/2408.12188/) (2024)
- [LLM Embeddings Improve Test-time Adaptation to Tabular Y|X-Shifts](https://mlsanity.com/paper/2410.07395/) (2024)
- [Embedding Domain-Specific Knowledge from LLMs into the Feature Engineering Pipeline](https://mlsanity.com/paper/2503.21155/) (2025)
- [Tab-MIA: A Benchmark Dataset for Membership Inference Attacks on Tabular Data in LLMs](https://mlsanity.com/paper/2507.17259/) (2025)
- [LLM Empowered Prototype Learning for Zero and Few-Shot Tasks on Tabular Data](https://mlsanity.com/paper/2508.09263/) (2025)
- [Large Language Models as Attribution Regularizers for Efficient Model Training](https://mlsanity.com/paper/2502.20268/) (2025)
- [ReSS: Learning Reasoning Models for Tabular Data Prediction via Symbolic Scaffold](https://mlsanity.com/paper/2604.13392/) (2026)
- [Harnessing LLMs Explanations to Boost Surrogate Models in Tabular Data Classification](https://mlsanity.com/paper/2505.05744/) (2025)
- [Data Curation Matters: Model Collapse and Spurious Shift Performance Prediction from Training on Uncurated Text Embeddings](https://mlsanity.com/paper/2506.17989/) (2025)
- [SMITE: Enhancing Fairness in LLMs through Optimal In-Context Example Selection via Dynamic Validation](https://mlsanity.com/paper/2508.17735/) (2025)
- [Summarize-Exemplify-Reflect: Data-driven Insight Distillation Empowers LLMs for Few-shot Tabular Classification](https://mlsanity.com/paper/2508.21561/) (2025)
- [Do You Even Need More Data? Rethinking LLM Deployment for Label-Scarce Tabular Classification with TabLens](https://mlsanity.com/paper/2605.24417/) (2026)
- [TAROT: Task-Adaptive Refinement of LLM-prior Graphs for Few-shot Tabular Learning](https://mlsanity.com/paper/2606.11640/) (2026)
- [Marginal Response Surface Elicitation for Zero-Label Tabular Learning](https://mlsanity.com/paper/2609.39639/) (2026)

## Abstract (arXiv)

Acquiring high-quality data is often a significant challenge in training machine learning (ML) models for tabular prediction, particularly in privacy-sensitive and costly domains like medicine and finance. Providing natural language instructions to large language models (LLMs) offers an alternative solution. However, it is unclear how effectively instructions leverage the knowledge in LLMs for solving tabular prediction problems. To address this gap, we introduce TABLET, a benchmark of 20 diverse tabular datasets annotated with instructions that vary in their phrasing, granularity, and technicality. Additionally, TABLET includes the instructions' logic and structured modifications to the instructions. We find in-context instructions increase zero-shot F1 performance for Flan-T5 11b by 44% on average and 13% for ChatGPT on TABLET. Also, we explore the limitations of using LLMs for tabular prediction in our benchmark by evaluating instruction faithfulness. We find LLMs often ignore instructions and fail to predict specific instances correctly, even with examples. Our analysis on TABLET shows that, while instructions help LLM performance, learning from instructions for tabular data requires new capabilities.