# An Automatic Prompt Generation System for Tabular Data Tasks

arXiv: https://arxiv.org/abs/2405.05618 · HTML: https://mlsanity.com/paper/2405.05618/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 34 references that are in the map.
- First posted on arXiv 9 May 2024.

## Builds on (most cited first, 16 of 16)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [Reinforcement Learning with Deep Energy-Based Policies](https://mlsanity.com/paper/1702.08165/) (2017)
- [SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training](https://mlsanity.com/paper/2106.01342/) (2021)
- [TABBIE: Pretrained Representations of Tabular Data](https://mlsanity.com/paper/2105.02584/) (2021)
- [TUTA: Tree-based Transformers for Generally Structured Table Pre-training](https://mlsanity.com/paper/2010.12537/) (2020)
- [Efficient (Soft) Q-Learning for Text Generation with Limited Good Data](https://mlsanity.com/paper/2106.07704/) (2021)
- [Trompt: Towards a Better Deep Neural Network for Tabular Data](https://mlsanity.com/paper/2305.18446/) (2023)
- [HoloDetect: Few-Shot Learning for Error Detection](https://mlsanity.com/paper/1904.02285/) (2019)
- [RPT: Relational Pre-trained Transformer Is Almost All You Need towards Democratizing Data Preparation](https://mlsanity.com/paper/2012.02469/) (2020)
- [Jellyfish: A Large Language Model for Data Preprocessing](https://mlsanity.com/paper/2312.01678/) (2023)

## Abstract (arXiv)

Efficient processing of tabular data is important in various industries, especially when working with datasets containing a large number of columns. Large language models (LLMs) have demonstrated their ability on several tasks through carefully crafted prompts. However, creating effective prompts for tabular datasets is challenging due to the structured nature of the data and the need to manage numerous columns. This paper presents an innovative auto-prompt generation system suitable for multiple LLMs, with minimal training. It proposes two novel methods; 1) A Reinforcement Learning-based algorithm for identifying and sequencing task-relevant columns 2) Cell-level similarity-based approach for enhancing few-shot example selection. Our approach has been extensively tested across 66 datasets, demonstrating improved performance in three downstream tasks: data imputation, error detection, and entity matching using two distinct LLMs; Google flan-t5-xxl and Mixtral 8x7B.