mlsanityData to 7 Oct 2026

From Zero to Hero: Advancing Zero-Shot Foundation Models for Tabular Outlier Detection

arXiv 2602.03018 ↗First posted 3 Feb 2026cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

Outlier detection (OD) is widely used in practice; but its effective deployment on new tasks is hindered by lack of labeled outliers, which makes algorithm and hyperparameter selection notoriously hard. Foundation models (FMs) have transformed ML, and OD is no exception: Shen et. al. (2025) introduced FoMo-0D, the first FM for OD, achieving remarkable performance against numerous baselines. This work introduces OUTFORMER, which advances FoMo-0D with (1) a mixture of synthetic priors and (2) self-evolving curriculum training. OUTFORMER is pretrained solely on synthetic labeled datasets and infers test labels of a new task by using its training data as in-context input. Inference is fast and zero-shot, requiring merely forward pass and no labeled outliers. Thanks to in-context learning, it requires zero additional work-no OD model training or bespoke model selection-enabling truly plug-and-play deployment. OUTFORMER achieves state-of-the-art performance on the prominent AdBench, as well as two new large-scale OD benchmarks that we introduce, comprising over 1,500 datasets, while maintaining speedy inference.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 58 of its 90 references that are in the map. First posted on arXiv 3 Feb 2026.

Builds on

58 of 90 references.

References · 58

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  5. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  6. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  7. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  8. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
  9. 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
  10. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  11. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 643
  12. 2016Training Region-based Object Detectors with Online Hard Example Miningcited by 577
38 more
  1. 2024Large Language Models: A Surveycited by 506
  2. 2019Which Tasks Should Be Learned Together in Multi-task Learning?cited by 367
  3. 2021Transformers Can Do Bayesian Inferencecited by 338
  4. 2025TabICL: A Tabular Foundation Model for In-Context Learning on Large Datacited by 267
  5. 2019Causality for Machine Learningcited by 242
  6. 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 234
  7. 2022ADBench: Anomaly Detection Benchmarkcited by 230
  8. 2019PyOD: A Python Toolbox for Scalable Outlier Detectioncited by 221
  9. 2023When Do Neural Nets Outperform Boosted Trees on Tabular Data?cited by 218
  10. 2020Classification-Based Anomaly Detection for General Datacited by 210
  11. 2017OpenML Benchmarking Suitescited by 169
  12. 2023AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Modelscited by 169
  13. 2021A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challengescited by 154
  14. 2025TabArena: A Living Benchmark for Machine Learning on Tabular Datacited by 152
  15. 2016Foundations of Structural Causal Models with Cycles and Latent Variablescited by 134
  16. 2025TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Modelscited by 115
  17. 2020When Do Curricula Work?cited by 105
  18. 2024Why Tabular Foundation Models Should Be a Research Prioritycited by 103
  19. 2022Denoising diffusion models for out-of-distribution detectioncited by 102
  20. 2023ForecastPFN: Synthetically-Trained Zero-Shot Forecastingcited by 102
  21. 2024TabDPT: Scaling Tabular Foundation Models on Real Datacited by 97
  22. 2019An Open Source AutoML Benchmarkcited by 85
  23. 2023On Diffusion Modeling for Anomaly Detectioncited by 71
  24. 2025Mitra: Mixed Synthetic Priors for Enhancing Tabular Foundation Modelscited by 67
  25. 2014Curriculum Learning of Multiple Taskscited by 65
  26. 2025LimiX: Unleashing Structured-Data Modeling Capability for Generalist Intelligencecited by 65
  27. 2024Is In-Context Learning in Large Language Models Bayesian? A Martingale Perspectivecited by 51
  28. 2024TuneTables: Context Optimization for Scalable Prior-Data Fitted Networkscited by 47
  29. 2024TALENT: A Tabular Analytics and Learning Toolboxcited by 46
  30. 2025Do-PFN: In-Context Learning for Causal Effect Estimationcited by 46
  31. 2025CausalPFN: Amortized Causal Effect Estimation via In-Context Learningcited by 43
  32. 2023TabRepo: A Large Scale Repository of Tabular Model Evaluations and its AutoML Applicationscited by 32
  33. 2025TimePFN: Effective Multivariate Time Series Forecasting with Synthetic Datacited by 27
  34. 2022Hyperparameter Sensitivity in Deep Outlier Detection: Analysis and a Scalable Hyper-Ensemble Solutioncited by 24
  35. 2023TabLib: A Dataset of 627M Tables with Contextcited by 23
  36. 2025AnomalyGFM: Graph Foundation Model for Zero/Few-shot Anomaly Detectioncited by 22
  37. 2024In-Context Data Distillation with TabPFNcited by 19
  38. 2022Toward Unsupervised Outlier Model Selectioncited by 18

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026MacrOData: New Benchmarks of Thousands of Datasets for Tabular Outlier Detectioncited by 5
  2. 2026VIP-COP: Context Optimization for Tabular Foundation Modelscited by 3
  3. 2026FoMo X: Modular Explainability Signals for Outlier Detection Foundation Modelscited by 1
  4. 2026Toward Privileged Foundation Models:LUPI for Accelerated and Improved Learningcited by 0
  5. 2026Why Ranking Anomaly Detection Algorithms Isn't as Reliable as You May Thinkcited by 0
  6. 2026When Identical Rows Disagree: From Benchmark Identifiability to Replication-Robust Anomaly Detectioncited by 0
  7. 2026When Less Compute Is More: Adaptive Early Exit Improves Pretrained Outlier Detectioncited by 0
  8. 2026TaskBridge: Bridging Unsupervised Tabular Anomaly Detection and In-Context Learning via Virtual Taskscited by 0