mlsanityData to 14 Oct 2026

Memory Efficient Tabular Foundation Models

arXiv 2607.27546 ↗First posted 30 Jul 2026cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Tabular Foundation Models, such as TabPFN, have received a large amount of recent attention due to their performance on in-context tabular machine learning tasks, which often exceeds classical baselines. However, practical deployment considerations of these models has received less attention. In this paper we investigate the memory requirements for these models. We demonstrate that employing model compression approaches can enable memory reductions of up to 7.6 with similar levels of performance, reducing deployment requirements by nearly 87%. Our work provides insight to practitioners seeking efficient deployment of these models in practical settings.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 32 of its 46 references that are in the map. First posted on arXiv 30 Jul 2026.

Builds on

32 of 46 references.

References · 32

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  3. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  4. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  5. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  6. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  7. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
  8. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  9. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
  10. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  11. 2021Tabular Data: Deep Learning is Not All You Needcited by 535
  12. 2014OpenML: networked science in machine learningcited by 526
20 more
  1. 2021Deep Neural Networks and Tabular Data: A Surveycited by 491
  2. 2024SpinQuant: LLM quantization with learned rotationscited by 471
  3. 20218-bit Optimizers via Block-wise Quantizationcited by 403
  4. 2021Transformers Can Do Bayesian Inferencecited by 360
  5. 2025TabICL: A Tabular Foundation Model for In-Context Learning on Large Datacited by 289
  6. 2024SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Modelscited by 170
  7. 2021Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Bettercited by 154
  8. 2026TabICLv2: A better, faster, scalable, and open tabular foundation modelcited by 144
  9. 2025TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Modelscited by 132
  10. 2025TurboQuant: Online Vector Quantization with Near-optimal Distortion Ratecited by 125
  11. 2024TabM: Advancing Tabular Deep Learning with Parameter-Efficient Ensemblingcited by 113
  12. 2024ServerlessLLM: Low-Latency Serverless Inference for Large Language Modelscited by 80
  13. 2025From Tables to Time: Extending TabPFN-v2 to Time Series Forecastingcited by 60
  14. 2025TabPFN Unleashed: A Scalable and Effective Solution to Tabular Classification Problemscited by 36
  15. 2025Representation Learning for Tabular Data: A Comprehensive Surveycited by 36
  16. 2025TabFlex: Scaling Tabular Learning to Millions with Linear Attentioncited by 15
  17. 2022Quantized Training of Gradient Boosting Decision Treescited by 6
  18. 2026End-to-End Compression for Tabular Foundation Modelscited by 6
  19. 2025TabDistill: Distilling Transformers into Neural Nets for Few-Shot Tabular Classificationcited by 3
  20. 2025Boosted Trees on a Diet: Compact Models for Resource-Constrained Devicescited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Attention Quantization for Tabular Foundation Modelscited by 0