# STUNT: Few-shot Tabular Learning with Self-generated Tasks from Unlabeled Tables

arXiv: https://arxiv.org/abs/2303.00918 · HTML: https://mlsanity.com/paper/2303.00918/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 59 references that are in the map.
- First posted on arXiv 2 Mar 2023.

## Builds on (most cited first, 30 of 41)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [Modeling Tabular data using Conditional GAN](https://mlsanity.com/paper/1907.00503/) (2019)
- [A Closer Look at Few-shot Classification](https://mlsanity.com/paper/1904.04232/) (2019)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [Tabular Data: Deep Learning is Not All You Need](https://mlsanity.com/paper/2106.03253/) (2021)
- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results](https://mlsanity.com/paper/1703.01780/) (2017)
- [Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?](https://mlsanity.com/paper/2003.11539/) (2020)
- [Meta Pseudo Labels](https://mlsanity.com/paper/2003.10580/) (2020)
- [True Few-Shot Learning with Language Models](https://mlsanity.com/paper/2105.11447/) (2021)
- [SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training](https://mlsanity.com/paper/2106.01342/) (2021)
- [Few-Shot Learning via Embedding Adaptation with Set-to-Set Functions](https://mlsanity.com/paper/1812.03664/) (2018)
- [Interpolation Consistency Training for Semi-Supervised Learning](https://mlsanity.com/paper/1903.03825/) (2019)
- [OpenML Benchmarking Suites](https://mlsanity.com/paper/1708.03731/) (2017)
- [Noisy Channel Language Model Prompting for Few-Shot Text Classification](https://mlsanity.com/paper/2108.04106/) (2021)
- [SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption](https://mlsanity.com/paper/2106.15147/) (2021)
- [Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samples](https://mlsanity.com/paper/2104.13963/) (2021)
- [Meta-Learning for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/1808.08437/) (2018)
- [Distribution Aligning Refinery of Pseudo-label for Imbalanced Semi-supervised Learning](https://mlsanity.com/paper/2007.08844/) (2020)
- [SubTab: Subsetting Features of Tabular Data for Self-Supervised Representation Learning](https://mlsanity.com/paper/2110.04361/) (2021)

## Built on by (most cited first, 30 of 49)

- [TabDPT: Scaling Tabular Foundation Models on Real Data](https://mlsanity.com/paper/2410.18164/) (2024)
- [Large Language Models Can Automatically Engineer Features for Few-Shot Tabular Learning](https://mlsanity.com/paper/2404.09491/) (2024)
- [Optimized Feature Generation for Tabular Data via LLMs with Decision Tree Reasoning](https://mlsanity.com/paper/2406.08527/) (2024)
- [LLM-FE: Automated Feature Engineering for Tabular Data with LLMs as Evolutionary Optimizers](https://mlsanity.com/paper/2503.14434/) (2025)
- [MediTab: Scaling Medical Tabular Data Predictors via Data Consolidation, Enrichment, and Refinement](https://mlsanity.com/paper/2305.12081/) (2023)
- [From Supervised to Generative: A Novel Paradigm for Tabular Deep Learning with Large Language Models](https://mlsanity.com/paper/2310.07338/) (2023)
- [A Performance-Driven Benchmark for Feature Selection in Tabular Deep Learning](https://mlsanity.com/paper/2311.05877/) (2023)
- [StructChart: On the Schema, Metric, and Augmentation for Visual Chart Understanding](https://mlsanity.com/paper/2309.11268/) (2023)
- [Language Modeling on Tabular Data: A Survey of Foundations, Techniques and Evolution](https://mlsanity.com/paper/2408.10548/) (2024)
- [A Survey on Self-Supervised Learning for Non-Sequential Tabular Data](https://mlsanity.com/paper/2402.01204/) (2024)
- [Graph Neural Networks for Tabular Data Learning: A Survey with Taxonomy and Directions](https://mlsanity.com/paper/2401.02143/) (2024)
- [CancerGPT: Few-shot Drug Pair Synergy Prediction using Large Pre-trained Language Models](https://mlsanity.com/paper/2304.10946/) (2023)
- [Tabular Transfer Learning via Prompting LLMs](https://mlsanity.com/paper/2408.11063/) (2024)
- [Zero-shot Meta-learning for Tabular Prediction Tasks with Adversarially Pre-trained Transformer](https://mlsanity.com/paper/2502.04573/) (2025)
- [Fine-Tuning the Retrieval Mechanism for Tabular Deep Learning](https://mlsanity.com/paper/2311.07343/) (2023)
- [FORESTLLM: Large Language Models Make Random Forest Great on Few-shot Tabular Learning](https://mlsanity.com/paper/2601.11311/) (2026)
- [Transfer Learning of Tabular Data by Finetuning Large Language Models](https://mlsanity.com/paper/2501.06863/) (2025)
- [Latte: Transfering LLMs` Latent-level Knowledge for Few-shot Tabular Learning](https://mlsanity.com/paper/2505.05237/) (2025)
- [A Pre-training Framework for Relational Data with Information-theoretic Principles](https://mlsanity.com/paper/2507.09837/) (2025)
- [TabSieve: Explicit In-Table Evidence Selection for Tabular Prediction](https://mlsanity.com/paper/2602.11700/) (2026)
- [Tabular Data Contrastive Learning via Class-Conditioned and Feature-Correlation Based Augmentation](https://mlsanity.com/paper/2404.17489/) (2024)
- [Data-Efficient Molecular Generation with Hierarchical Textual Inversion](https://mlsanity.com/paper/2405.02845/) (2024)
- [TabDistill: Distilling Transformers into Neural Nets for Few-Shot Tabular Classification](https://mlsanity.com/paper/2511.05704/) (2025)
- [Tabular Few-Shot Generalization Across Heterogeneous Feature Spaces](https://mlsanity.com/paper/2311.10051/) (2023)
- [TabularFM: An Open Framework For Tabular Foundational Models](https://mlsanity.com/paper/2406.09837/) (2024)
- [LLM Empowered Prototype Learning for Zero and Few-Shot Tasks on Tabular Data](https://mlsanity.com/paper/2508.09263/) (2025)
- [ReviewScore: Misinformed Peer Review Detection with Large Language Models](https://mlsanity.com/paper/2509.21679/) (2025)
- [TablEye: Seeing small Tables through the Lens of Images](https://mlsanity.com/paper/2307.02491/) (2023)
- [Pretrained deep models outperform GBDTs in Learning-To-Rank under label scarcity](https://mlsanity.com/paper/2308.00177/) (2023)
- [Large Language Models as Attribution Regularizers for Efficient Model Training](https://mlsanity.com/paper/2502.20268/) (2025)

## Abstract (arXiv)

Learning with few labeled tabular samples is often an essential requirement for industrial machine learning applications as varieties of tabular data suffer from high annotation costs or have difficulties in collecting new samples for novel tasks. Despite the utter importance, such a problem is quite under-explored in the field of tabular learning, and existing few-shot learning schemes from other domains are not straightforward to apply, mainly due to the heterogeneous characteristics of tabular data. In this paper, we propose a simple yet effective framework for few-shot semi-supervised tabular learning, coined Self-generated Tasks from UNlabeled Tables (STUNT). Our key idea is to self-generate diverse few-shot tasks by treating randomly chosen columns as a target label. We then employ a meta-learning scheme to learn generalizable knowledge with the constructed tasks. Moreover, we introduce an unsupervised validation scheme for hyperparameter search (and early stopping) by generating a pseudo-validation set using STUNT from unlabeled data. Our experimental results demonstrate that our simple framework brings significant performance gain under various tabular few-shot learning benchmarks, compared to prior semi- and self-supervised baselines. Code is available at https://github.com/jaehyun513/STUNT.