mlsanityData to 14 Oct 2026

On the Efficiency of NLP-Inspired Methods for Tabular Deep Learning

arXiv 2411.17207 ↗First posted 26 Nov 2024cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Recent advancements in tabular deep learning (DL) have led to substantial performance improvements, surpassing the capabilities of traditional models. With the adoption of techniques from natural language processing (NLP), such as language model-based approaches, DL models for tabular data have also grown in complexity and size. Although tabular datasets do not typically pose scalability issues, the escalating size of these models has raised efficiency concerns. Despite its importance, efficiency has been relatively underexplored in tabular DL research. This paper critically examines the latest innovations in tabular DL, with a dual focus on performance and computational efficiency. The source code is available at https://github.com/basf/mamba-tabular.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 20 of its 26 references that are in the map. First posted on arXiv 26 Nov 2024.

Builds on

20 of 26 references.

References · 20

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  3. 2020Longformer: The Long-Document Transformercited by 3,413
  4. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  5. 2021Revisiting Deep Learning Models for Tabular Datacited by 740
  6. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
  7. 2022MTEB: Massive Text Embedding Benchmarkcited by 640
  8. 2021Deep Neural Networks and Tabular Data: A Surveycited by 491
  9. 2020TabTransformer: Tabular Data Modeling Using Contextual Embeddingscited by 409
  10. 2024Jamba: A Hybrid Transformer-Mamba Language Modelcited by 401
  11. 2019InterpretML: A Unified Framework for Machine Learning Interpretabilitycited by 246
  12. 2023When Do Neural Nets Outperform Boosted Trees on Tabular Data?cited by 223
8 more
  1. 2022On Embeddings for Numerical Features in Tabular Deep Learningcited by 186
  2. 2024Graph Mamba: Towards Learning on Graphs with State Space Modelscited by 86
  3. 2021Video Summarization Using Deep Neural Networks: A Surveycited by 85
  4. 2024Point Mamba: A Novel Point Cloud Backbone Based on State Space Model with Octree-Based Ordering Strategycited by 73
  5. 2024TimeMachine: A Time Series is Worth 4 Mambas for Long-term Forecastingcited by 64
  6. 2024MoE-Mamba: Efficient Selective State Space Models with Mixture of Expertscited by 60
  7. 2023Deep Learning-based 3D Point Cloud Classification: A Systematic Survey and Outlookcited by 24
  8. 2024Mambular: A Sequential Model for Tabular Deep Learningcited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 5.

7 papers.

  1. 2026Mitigating Label Shift in Tabular In-Context Learning via Test-Time Posterior Adjustmentcited by 4
  2. 2025Beyond Black-Box Predictions: Identifying Marginal Feature Effects in Tabular Transformer Networkscited by 2
  3. 2026TabPrep: Closing the Feature Engineering Gap in Tabular Benchmarkscited by 2
  4. 2026ZAYAN: Disentangled Contrastive Transformer for Tabular Remote Sensing Datacited by 1
  5. 2026DynaTab: Dynamic Feature Ordering as Neural Rewiring for High-Dimensional Tabular Datacited by 1
  6. 2025Aeolus: A Multi-structural Flight Delay Datasetcited by 0
  7. 2026An approach with Visual and Tabular Mamba to multimodal medical data using Mixed Fusioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024TabM: Advancing Tabular Deep Learning with Parameter-Efficient Ensembling
  2. 2023TabR: Tabular Deep Learning Meets Nearest Neighbors in 2023
  3. 2026Why Large Language Models Fail at Tabular Prediction
  4. 2020Tabular GANs for uneven distribution
  5. 2025Make Still Further Progress: Chain of Thoughts for Tabular Data Leaderboard
  6. 2024TabDPT: Scaling Tabular Foundation Models on Real Data
  7. 2025Unveiling the Role of Data Uncertainty in Tabular Deep Learning
  8. 2023TabuLa: Harnessing Language Models for Tabular Data Synthesis