# Trompt: Towards a Better Deep Neural Network for Tabular Data

arXiv: https://arxiv.org/abs/2305.18446 · HTML: https://mlsanity.com/paper/2305.18446/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 40 references that are in the map.
- First posted on arXiv 29 May 2023.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Decision Transformer: Reinforcement Learning via Sequence Modeling](https://mlsanity.com/paper/2106.01345/) (2021)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [TabNet: Attentive Interpretable Tabular Learning](https://mlsanity.com/paper/1908.07442/) (2019)
- [Tabular Data: Deep Learning is Not All You Need](https://mlsanity.com/paper/2106.03253/) (2021)
- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [Deep Neural Networks and Tabular Data: A Survey](https://mlsanity.com/paper/2110.01889/) (2021)
- [TabTransformer: Tabular Data Modeling Using Contextual Embeddings](https://mlsanity.com/paper/2012.06678/) (2020)
- [Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates](https://mlsanity.com/paper/1610.00633/) (2016)
- [SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training](https://mlsanity.com/paper/2106.01342/) (2021)
- [Learning to Explain: An Information-Theoretic Perspective on Model Interpretation](https://mlsanity.com/paper/1802.07814/) (2018)
- [Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Loss](https://mlsanity.com/paper/2002.02562/) (2020)
- [Online Deep Learning: Learning Deep Neural Networks on the Fly](https://mlsanity.com/paper/1711.03705/) (2017)

## Built on by (most cited first, 30 of 36)

- [TabM: Advancing Tabular Deep Learning with Parameter-Efficient Ensembling](https://mlsanity.com/paper/2410.24210/) (2024)
- [Relational Deep Learning: Graph Representation Learning on Relational Databases](https://mlsanity.com/paper/2312.04615/) (2023)
- [TALENT: A Tabular Analytics and Learning Toolbox](https://mlsanity.com/paper/2407.04057/) (2024)
- [PyG 2.0: Scalable Learning on Real World Graphs](https://mlsanity.com/paper/2507.16991/) (2025)
- [Optimized Feature Generation for Tabular Data via LLMs with Decision Tree Reasoning](https://mlsanity.com/paper/2406.08527/) (2024)
- [TabPFN Unleashed: A Scalable and Effective Solution to Tabular Classification Problems](https://mlsanity.com/paper/2502.02527/) (2025)
- [Representation Learning for Tabular Data: A Comprehensive Survey](https://mlsanity.com/paper/2504.16109/) (2025)
- [TabReD: Analyzing Pitfalls and Filling the Gaps in Tabular Deep Learning Benchmarks](https://mlsanity.com/paper/2406.19380/) (2024)
- [Relational Deep Learning: Challenges, Foundations and Next-Generation Architectures](https://mlsanity.com/paper/2506.16654/) (2025)
- [A Survey on Deep Tabular Learning](https://mlsanity.com/paper/2410.12034/) (2024)
- [A Data-Centric Perspective on Evaluating Machine Learning Models for Tabular Data](https://mlsanity.com/paper/2407.02112/) (2024)
- [Binning as a Pretext Task: Improving Self-Supervised Learning in Tabular Domains](https://mlsanity.com/paper/2405.07414/) (2024)
- [Transformers Meet Relational Databases](https://mlsanity.com/paper/2412.05218/) (2024)
- [LLM Meeting Decision Trees on Tabular Data](https://mlsanity.com/paper/2505.17918/) (2025)
- [REDELEX: A Framework for Relational Deep Learning Exploration](https://mlsanity.com/paper/2506.22199/) (2025)
- [TabKANet: Tabular Data Modeling with Kolmogorov-Arnold Network and Transformer](https://mlsanity.com/paper/2409.08806/) (2024)
- [Task-Agnostic Contrastive Pretraining for Relational Deep Learning](https://mlsanity.com/paper/2506.22530/) (2025)
- [A Pre-training Framework for Relational Data with Information-theoretic Principles](https://mlsanity.com/paper/2507.09837/) (2025)
- [Retrieval-aligned Tabular Foundation Models Enable Robust Clinical Risk Prediction in Electronic Health Records Under Real-world Constraints](https://mlsanity.com/paper/2604.01841/) (2026)
- [Tree-Regularized Tabular Embeddings](https://mlsanity.com/paper/2403.00963/) (2024)
- [Make Still Further Progress: Chain of Thoughts for Tabular Data Leaderboard](https://mlsanity.com/paper/2505.13421/) (2025)
- [LakeMLB: Data Lake Machine Learning Benchmark](https://mlsanity.com/paper/2602.10441/) (2026)
- [TabMixer: Noninvasive Estimation of the Mean Pulmonary Artery Pressure via Imaging and Tabular Data Mixing](https://mlsanity.com/paper/2409.07564/) (2024)
- [RCCFormer: A Robust Crowd Counting Network Based on Transformer](https://mlsanity.com/paper/2504.04935/) (2025)
- [Deep Tabular Representation Corrector](https://mlsanity.com/paper/2603.16569/) (2026)
- [DynaTab: Dynamic Feature Ordering as Neural Rewiring for High-Dimensional Tabular Data](https://mlsanity.com/paper/2605.03430/) (2026)
- [MaskTab: Scalable Masked Tabular Pretraining with Scaling Laws and Distillation for Industrial Classification](https://mlsanity.com/paper/2605.11408/) (2026)
- [An Automatic Prompt Generation System for Tabular Data Tasks](https://mlsanity.com/paper/2405.05618/) (2024)
- [Table2Image: Lightweight Tabular Learning with Generated Proxy Representations and Reliability Diagnostics](https://mlsanity.com/paper/2412.06265/) (2024)
- [Multi-branch of Attention Yields Accurate Results for Tabular Data](https://mlsanity.com/paper/2502.12507/) (2025)

## Abstract (arXiv)

Tabular data is arguably one of the most commonly used data structures in various practical domains, including finance, healthcare and e-commerce. The inherent heterogeneity allows tabular data to store rich information. However, based on a recently published tabular benchmark, we can see deep neural networks still fall behind tree-based models on tabular datasets. In this paper, we propose Trompt--which stands for Tabular Prompt--a novel architecture inspired by prompt learning of language models. The essence of prompt learning is to adjust a large pre-trained model through a set of prompts outside the model without directly modifying the model. Based on this idea, Trompt separates the learning strategy of tabular data into two parts. The first part, analogous to pre-trained models, focus on learning the intrinsic information of a table. The second part, analogous to prompts, focus on learning the variations among samples. Trompt is evaluated with the benchmark mentioned above. The experimental results demonstrate that Trompt outperforms state-of-the-art deep neural networks and is comparable to tree-based models.