# From Zero to Hero: Advancing Zero-Shot Foundation Models for Tabular Outlier Detection

arXiv: https://arxiv.org/abs/2602.03018 · HTML: https://mlsanity.com/paper/2602.03018/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 58 of its 90 references that are in the map.
- First posted on arXiv 3 Feb 2026.

## Builds on (most cited first, 30 of 58)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [An Explanation of In-context Learning as Implicit Bayesian Inference](https://mlsanity.com/paper/2111.02080/) (2021)
- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [Generalized Out-of-Distribution Detection: A Survey](https://mlsanity.com/paper/2110.11334/) (2021)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022)
- [Training Region-based Object Detectors with Online Hard Example Mining](https://mlsanity.com/paper/1604.03540/) (2016)
- [Large Language Models: A Survey](https://mlsanity.com/paper/2402.06196/) (2024)
- [Which Tasks Should Be Learned Together in Multi-task Learning?](https://mlsanity.com/paper/1905.07553/) (2019)
- [Transformers Can Do Bayesian Inference](https://mlsanity.com/paper/2112.10510/) (2021)
- [TabICL: A Tabular Foundation Model for In-Context Learning on Large Data](https://mlsanity.com/paper/2502.05564/) (2025)
- [Causality for Machine Learning](https://mlsanity.com/paper/1911.10500/) (2019)
- [Foundation Models for Time Series Analysis: A Tutorial and Survey](https://mlsanity.com/paper/2403.14735/) (2024)
- [ADBench: Anomaly Detection Benchmark](https://mlsanity.com/paper/2206.09426/) (2022)
- [When Do Neural Nets Outperform Boosted Trees on Tabular Data?](https://mlsanity.com/paper/2305.02997/) (2023)
- [PyOD: A Python Toolbox for Scalable Outlier Detection](https://mlsanity.com/paper/1901.01588/) (2019)
- [Classification-Based Anomaly Detection for General Data](https://mlsanity.com/paper/2005.02359/) (2020)
- [OpenML Benchmarking Suites](https://mlsanity.com/paper/1708.03731/) (2017)
- [TabArena: A Living Benchmark for Machine Learning on Tabular Data](https://mlsanity.com/paper/2506.16791/) (2025)
- [AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models](https://mlsanity.com/paper/2308.15366/) (2023)
- [A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challenges](https://mlsanity.com/paper/2110.14051/) (2021)
- [Foundations of Structural Causal Models with Cycles and Latent Variables](https://mlsanity.com/paper/1611.06221/) (2016)
- [TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Models](https://mlsanity.com/paper/2511.08667/) (2025)
- [TabDPT: Scaling Tabular Foundation Models on Real Data](https://mlsanity.com/paper/2410.18164/) (2024)
- [When Do Curricula Work?](https://mlsanity.com/paper/2012.03107/) (2020)

## Built on by (most cited first, 9 of 9)

- [MacrOData: New Benchmarks of Thousands of Datasets for Tabular Outlier Detection](https://mlsanity.com/paper/2602.09329/) (2026)
- [VIP-COP: Context Optimization for Tabular Foundation Models](https://mlsanity.com/paper/2605.12904/) (2026)
- [FoMo X: Modular Explainability Signals for Outlier Detection Foundation Models](https://mlsanity.com/paper/2603.17570/) (2026)
- [Toward Privileged Foundation Models:LUPI for Accelerated and Improved Learning](https://mlsanity.com/paper/2605.07799/) (2026)
- [CORE: In-Context Reconstruction for Unified Tabular Anomaly Detection](https://mlsanity.com/paper/2607.27615/) (2026)
- [Why Ranking Anomaly Detection Algorithms Isn't as Reliable as You May Think](https://mlsanity.com/paper/2608.04613/) (2026)
- [When Identical Rows Disagree: From Benchmark Identifiability to Replication-Robust Anomaly Detection](https://mlsanity.com/paper/2609.29580/) (2026)
- [When Less Compute Is More: Adaptive Early Exit Improves Pretrained Outlier Detection](https://mlsanity.com/paper/2609.32898/) (2026)
- [TaskBridge: Bridging Unsupervised Tabular Anomaly Detection and In-Context Learning via Virtual Tasks](https://mlsanity.com/paper/2609.36968/) (2026)

## Abstract (arXiv)

Outlier detection (OD) is widely used in practice; but its effective deployment on new tasks is hindered by lack of labeled outliers, which makes algorithm and hyperparameter selection notoriously hard. Foundation models (FMs) have transformed ML, and OD is no exception: Shen et. al. (2025) introduced FoMo-0D, the first FM for OD, achieving remarkable performance against numerous baselines. This work introduces OUTFORMER, which advances FoMo-0D with (1) a mixture of synthetic priors and (2) self-evolving curriculum training. OUTFORMER is pretrained solely on synthetic labeled datasets and infers test labels of a new task by using its training data as in-context input. Inference is fast and zero-shot, requiring merely forward pass and no labeled outliers. Thanks to in-context learning, it requires zero additional work-no OD model training or bespoke model selection-enabling truly plug-and-play deployment. OUTFORMER achieves state-of-the-art performance on the prominent AdBench, as well as two new large-scale OD benchmarks that we introduce, comprising over 1,500 datasets, while maintaining speedy inference.