# PACE: Plug-and-Play Contextual Embedding for Feature Screening with Pretrained Tabular Foundation Models

arXiv: https://arxiv.org/abs/2609.23574 · HTML: https://mlsanity.com/paper/2609.23574/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 60 references that are in the map.
- First posted on arXiv 20 Sep 2026.

## Builds on (most cited first, 30 of 30)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Distribution-Free Predictive Inference For Regression](https://mlsanity.com/paper/1604.04173/) (2016)
- [TabICL: A Tabular Foundation Model for In-Context Learning on Large Data](https://mlsanity.com/paper/2502.05564/) (2025)
- [TabICLv2: A better, faster, scalable, and open tabular foundation model](https://mlsanity.com/paper/2602.11139/) (2026)
- [TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Models](https://mlsanity.com/paper/2511.08667/) (2025)
- [TabDPT: Scaling Tabular Foundation Models on Real Data](https://mlsanity.com/paper/2410.18164/) (2024)
- [TabM: Advancing Tabular Deep Learning with Parameter-Efficient Ensembling](https://mlsanity.com/paper/2410.24210/) (2024)
- [Better by Default: Strong Pre-Tuned MLPs and Boosted Trees on Tabular Data](https://mlsanity.com/paper/2407.04491/) (2024)
- [LassoNet: A Neural Network with Feature Sparsity](https://mlsanity.com/paper/1907.12207/) (2019)
- [TALENT: A Tabular Analytics and Learning Toolbox](https://mlsanity.com/paper/2407.04057/) (2024)
- [A Closer Look at TabPFN v2: Understanding Its Strengths and Extending Its Capabilities](https://mlsanity.com/paper/2502.17361/) (2025)
- [A Performance-Driven Benchmark for Feature Selection in Tabular Deep Learning](https://mlsanity.com/paper/2311.05877/) (2023)
- [Beyond IID: How General Are Tabular Foundation Models, Really?](https://mlsanity.com/paper/2606.30410/) (2026)
- [Scaling TabPFN: Sketching and Feature Selection for Tabular Prior-Data Fitted Networks](https://mlsanity.com/paper/2311.10609/) (2023)
- [xRFM: Accurate, scalable, and interpretable feature learning models for tabular data](https://mlsanity.com/paper/2508.10053/) (2025)
- [TabPFN-Wide: Continued Pre-Training for Extreme Feature Counts](https://mlsanity.com/paper/2510.06162/) (2025)
- [Model-free Feature Screening and FDR Control with Knockoff Features](https://mlsanity.com/paper/1908.06597/) (2019)
- [The fused Kolmogorov filter: A nonparametric model-free screening method](https://mlsanity.com/paper/1403.7701/) (2014)
- [Deep Feature Screening: Feature Selection for Ultra High-Dimensional Data via Deep Neural Networks](https://mlsanity.com/paper/2204.01682/) (2022)
- [Towards Universal Tabular Embeddings: A Benchmark Across Data Tasks](https://mlsanity.com/paper/2604.21696/) (2026)
- [ExplainerPFN: Towards tabular foundation models for model-free zero-shot feature importance estimations](https://mlsanity.com/paper/2601.23068/) (2026)
- [Mitra-v2 Technical Report](https://mlsanity.com/paper/2609.04540/) (2026)
- [Air-HOLP: Adaptive Regularized Feature Screening for High Dimensional Correlated Data](https://mlsanity.com/paper/2408.13000/) (2024)
- [Comparing Task-Agnostic Embedding Models for Tabular Data](https://mlsanity.com/paper/2511.14276/) (2025)
- [Beyond Black-Box Predictions: Identifying Marginal Feature Effects in Tabular Transformer Networks](https://mlsanity.com/paper/2504.08712/) (2025)
- [GOTabPFN: From Feature Ordering to Compact Tokenization for Tabular Foundation Models on High-Dimensional Data](https://mlsanity.com/paper/2606.05441/) (2026)
- [SAND: One-Shot Feature Selection with Additive Noise Distortion](https://mlsanity.com/paper/2505.03923/) (2025)
- [Label-shift robust federated feature screening for high-dimensional classification](https://mlsanity.com/paper/2506.00379/) (2025)
- [Knockoffs Inference under Privacy Constraints](https://mlsanity.com/paper/2506.09690/) (2025)

## Abstract (arXiv)

In high-dimensional tabular learning, feature screening provides a lightweight, model-agnostic way to remove irrelevant features before model fitting. However, scoring raw values directly can miss nonlinear or distributional structure. We introduce PACE (Plug-and-Play Contextual Embedding), which inserts a frozen tabular foundation model (TFM) column encoder before an existing feature-scoring rule, expanding each feature into a higher-dimensional contextual representation. Across controlled studies, PACE improves raw-space screening of complex nonlinear dependence with only modest additional encoding cost. These gains translate to downstream prediction on TALENT datasets: PACE-DC improves binary AUC by 0.077 and multiclass macro-AUC by 0.064, with a median normalized RMSE improvement of 0.063 across ten learners. Matched random-weight and random-feature controls show that PACE gains from pretrained structure beyond generic dimensional expansion. PACE further achieves favorable performance--time trade-offs against task-fitted selectors and attribution-based methods, positioning pretrained column geometry as a reusable upstream primitive for tabular learning.