mlsanityData to 14 Oct 2026

Scaling TabPFN: Sketching and Feature Selection for Tabular Prior-Data Fitted Networks

arXiv 2311.10609 ↗First posted 17 Nov 2023cs.LG cs.DBcited by 18

Not on the map. Why?

Abstract (arXiv)

Tabular classification has traditionally relied on supervised algorithms, which estimate the parameters of a prediction model using its training data. Recently, Prior-Data Fitted Networks (PFNs) such as TabPFN have successfully learned to classify tabular data in-context: the model parameters are designed to classify new samples based on labelled training samples given after the model training. While such models show great promise, their applicability to real-world data remains limited due to the computational scale needed. Here we study the following question: given a pre-trained PFN for tabular data, what is the best way to summarize the labelled training samples before feeding them to the model? We conduct an initial investigation of sketching and feature-selection methods for TabPFN, and note certain key differences between it and conventionally fitted tabular models.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 20 of its 31 references that are in the map. First posted on arXiv 17 Nov 2023.

Builds on

20 of 31 references.

References · 20

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  3. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  4. 2017Billion-scale similarity search with GPUscited by 2,497
  5. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  6. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  7. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 938
  8. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
  9. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 740
  10. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
  11. 2021Deep Neural Networks and Tabular Data: A Surveycited by 491
  12. 2021SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Trainingcited by 319
8 more
  1. 2019Towards Efficient Data Valuation Based on the Shapley Valuecited by 281
  2. 2023When Do Neural Nets Outperform Boosted Trees on Tabular Data?cited by 223
  3. 2022Active Learning on a Budget: Opposite Strategies Suit High and Low Budgetscited by 133
  4. 2022A Length-Extrapolatable Transformercited by 127
  5. 2023HyperAttention: Long-context Attention in Near-Linear Timecited by 85
  6. 2023Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Taskscited by 84
  7. 2015A Review of Feature Selection Methods Based on Mutual Informationcited by 67
  8. 2022Active Learning Through a Covering Lenscited by 66

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 6; 2026: 6.

18 papers.

Sort:

  1. 2025A Closer Look at TabPFN v2: Understanding Its Strengths and Extending Its Capabilitiescited by 41
  2. 2025TabPFN Unleashed: A Scalable and Effective Solution to Tabular Classification Problemscited by 36
  3. 2025Representation Learning for Tabular Data: A Comprehensive Surveycited by 36
  4. 2024Mixture of In-Context Prompters for Tabular PFNscited by 33
  5. 2024In-Context Data Distillation with TabPFNcited by 19
  6. 2025Effortless, Simulation-Efficient Bayesian Inference using Tabular Foundation Modelscited by 13
  7. 2024LaT-PFN: A Joint Embedding Predictive Architecture for In-context Time-series Forecastingcited by 8
  8. 2024Learning a Decision Tree Algorithm with Transformerscited by 6
  9. 2024PMLBmini: A Tabular Classification Benchmark Suite for Data-Scarce Applicationscited by 6
  10. 2025From Tables to Signals: Revealing Spectral Adaptivity in TabPFNcited by 4
  11. 2026VIP-COP: Context Optimization for Tabular Foundation Modelscited by 3
  12. 2026CRUMB: Efficient Prior Fitted Network Inference via Distributionally Matched Context Batchingcited by 2
6 more
  1. 2024Squeezing Lemons with Hammers: An Evaluation of AutoML and Tabular Deep Learning for Data-Scarce Classification Applicationscited by 1
  2. 2025Class-Imbalanced-Aware Adaptive Dataset Distillation for Scalable Pretrained Model on Credit Scoringcited by 1
  3. 2026uLEAD-TabPFN: Uncertainty-aware Dependency-based Anomaly Detection with TabPFNcited by 1
  4. 2026Streaming Hierarchical Inference with Tabular Foundation Modelscited by 1
  5. 2026PACE: Plug-and-Play Contextual Embedding for Feature Screening with Pretrained Tabular Foundation Modelscited by 0
  6. 2026Compile the Table: Query-Calibrated Operator Compression for Tabular In-Context Learningcited by 0