mlsanityData to 7 Oct 2026

LLM Embeddings for Deep Learning on Tabular Data

arXiv 2502.11596 ↗First posted 17 Feb 2025cs.LG cs.AIcited by 11

Not on the map. Why?

Abstract (arXiv)

Tabular deep-learning methods require embedding numerical and categorical input features into high-dimensional spaces before processing them. Existing methods deal with this heterogeneous nature of tabular data by employing separate type-specific encoding approaches. This limits the cross-table transfer potential and the exploitation of pre-trained knowledge. We propose a novel approach that first transforms tabular data into text, and then leverages pre-trained representations from LLMs to encode this data, resulting in a plug-and-play solution to improv ing deep-learning tabular methods. We demonstrate that our approach improves accuracy over competitive models, such as MLP, ResNet and FT-Transformer, by validating on seven classification datasets.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 38 of its 48 references that are in the map. First posted on arXiv 17 Feb 2025.

Builds on

38 of 48 references.

References · 38

  1. 2017Attention Is All You Needcited by 59,932
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  6. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
  7. 2021Revisiting Deep Learning Models for Tabular Datacited by 725
  8. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 643
  9. 2019TabNet: Attentive Interpretable Tabular Learningcited by 640
  10. 2022MTEB: Massive Text Embedding Benchmarkcited by 623
  11. 2021Deep Neural Networks and Tabular Data: A Surveycited by 489
  12. 2020TabTransformer: Tabular Data Modeling Using Contextual Embeddingscited by 398
26 more
  1. 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 396
  2. 2022TabLLM: Few-shot Classification of Tabular Data with Large Language Modelscited by 273
  3. 2020Byte Pair Encoding is Suboptimal for Language Model Pretrainingcited by 189
  4. 2022On Embeddings for Numerical Features in Tabular Deep Learningcited by 185
  5. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 157
  6. 2022LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Taskscited by 146
  7. 2022TransTab: Learning Transferable Tabular Transformers Across Tablescited by 132
  8. 2024Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Modelscited by 119
  9. 2016Time for a change: a tutorial for comparing multiple classifiers through Bayesian analysiscited by 92
  10. 2023Scaling Sentence Embeddings with Large Language Modelscited by 83
  11. 2024Large Language Models Can Automatically Engineer Features for Few-Shot Tabular Learningcited by 79
  12. 2024Making Pre-trained Language Models Great on Tabular Predictioncited by 56
  13. 2023Generative Table Pre-training Empowers Models for Tabular Predictioncited by 55
  14. 2024Elephants Never Forget: Memorization and Learning of Tabular Data in Large Language Modelscited by 34
  15. 2024Mambular: A Sequential Model for Tabular Deep Learningcited by 18
  16. 2024BGE Landmark Embedding: A Chunking-Free Embedding Method For Retrieval Augmented Long-Context Large Language Modelscited by 15
  17. 2024Binning as a Pretext Task: Improving Self-Supervised Learning in Tabular Domainscited by 12
  18. 2020Propositionalization and Embeddings: Two Sides of the Same Coincited by 8
  19. 2024Tabular Embedding Model (TEM): Finetuning Embedding Models For Tabular RAG Applicationscited by 8
  20. 2024Deep Feature Embedding for Tabular Datacited by 7
  21. 2022GCondNet: A Novel Method for Improving Neural Networks on Small High-Dimensional Tabular Datacited by 6
  22. 2024TabMDA: Tabular Manifold Data Augmentation for Any Classifier using Transformers with In-context Subsettingcited by 5
  23. 2024Tree-Regularized Tabular Embeddingscited by 3
  24. 2023Enabling tabular deep learning when d gg n with an auxiliary knowledge graphcited by 2
  25. 2023Latent Graphs for Semi-Supervised Learning on Biomedical Tabular Datacited by 1
  26. 2023Incorporating LLM Priors into Tabular Learnerscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 7.

11 papers.

  1. 2026Eliciting Numerical Predictive Distributions of LLMs Without Autoregressioncited by 5
  2. 2025AntiDote: Bi-level Adversarial Training for Tamper-Resistant LLMscited by 4
  3. 2025Exploring the Capabilities of Large Language Model Encoders for Image-Text Retrieval in Chest X-rayscited by 1
  4. 2026Language Model Representations for Efficient Few-Shot Tabular Classificationcited by 1
  5. 2026Schema-Adaptive Tabular Representation Learning with LLMs for Generalizable Multimodal Clinical Reasoningcited by 1
  6. 2025Universal Laboratory Model: prognosis of abnormal clinical outcomes based on routine testscited by 0
  7. 2025Innovative tokenisation of structured data for LLM trainingcited by 0
  8. 2026Embedding World Knowledge into Tabular Models: Towards Best Practices for Embedding Pipeline Designcited by 0
  9. 2026MAPLE: Multi-Path Adaptive Propagation with Level-Aware Embeddings for Hierarchical Multi-Label Image Classificationcited by 0
  10. 2026Predicting Decisions of AI Agents from Limited Interaction through Text-Tabular Modelingcited by 0
  11. 2026TabNSM: Neural Sparse Mixer for Tabular Regressioncited by 0