mlsanityData to 7 Oct 2026

Mambular: A Sequential Model for Tabular Deep Learning

arXiv 2408.06291 ↗First posted 12 Aug 2024cs.LGcited by 18

On the map: State-space sequence models (S4, Mamba) → Core SSM architectures for language

Abstract (arXiv)

The analysis of tabular data has traditionally been dominated by gradient-boosted decision trees (GBDTs), known for their proficiency with mixed categorical and numerical features. However, recent deep learning innovations are challenging this dominance. This paper investigates the use of autoregressive state-space models for tabular data and compares their performance against established benchmark models. Additionally, we explore various adaptations of these models, including different pooling strategies, feature interaction mechanisms, and bi-directional processing techniques to understand their effectiveness for tabular data. Our findings indicate that interpreting features as a sequence and processing them and their interactions through structured state-space layers can lead to significant performance improvement. This research underscores the versatility of autoregressive models in tabular data analysis, positioning them as a promising alternative that could substantially enhance deep learning capabilities in this traditionally challenging area. The source code is available at https://github.com/basf/mamba-tabular.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Placed in Core SSM architectures for language, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 24 of its 46 references that are in the map. First posted on arXiv 12 Aug 2024.

Builds on

24 of 46 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Jamba: A Hybrid Transformer-Mamba Language Model28 Mar
  2. 2024MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts8 Jan
  3. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spaces1 Dec
  4. 2021Efficiently Modeling Long Sequences with Structured State Spaces31 Oct

Other references · 20

  1. 2021Revisiting Deep Learning Models for Tabular Datacited by 725
  2. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 643
  3. 2019TabNet: Attentive Interpretable Tabular Learningcited by 640
  4. 2021Deep Neural Networks and Tabular Data: A Surveycited by 489
  5. 2020TabTransformer: Tabular Data Modeling Using Contextual Embeddingscited by 398
  6. 2023When Do Neural Nets Outperform Boosted Trees on Tabular Data?cited by 218
  7. 2019Neural Oblivious Decision Ensembles for Deep Learning on Tabular Datacited by 200
  8. 2022On Embeddings for Numerical Features in Tabular Deep Learningcited by 185
  9. 2022TransTab: Learning Transferable Tabular Transformers Across Tablescited by 132
  10. 2024Is Mamba Effective for Time Series Forecasting?cited by 110
  11. 2024Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modelingcited by 108
  12. 2024Cobra: Extending Mamba to Multi-Modal Large Language Model for Efficient Inferencecited by 97
8 more
  1. 2024MedMamba: Vision Mamba for Medical Image Classificationcited by 86
  2. 2024Graph Mamba: Towards Learning on Graphs with State Space Modelscited by 84
  3. 2024SiMBA: Simplified Mamba-Based Architecture for Vision and Multivariate Time seriescited by 81
  4. 2024Point Mamba: A Novel Point Cloud Backbone Based on State Space Model with Octree-Based Ordering Strategycited by 71
  5. 2024TimeMachine: A Time Series is Worth 4 Mambas for Long-term Forecastingcited by 63
  6. 2023Neural Additive Models for Location Scale and Shape: A Framework for Interpretable Neural Regression Beyond the Meancited by 18
  7. 2019XGBoostLSS -- An extension of XGBoost to probabilistic forecastingcited by 11
  8. 2022Distributional Gradient Boosting Machinescited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 9; 2026: 6.

18 papers.

Sort:

By area:

  1. 2025ConTextTab: A Semantics-Aware Tabular In-Context Learnercited by 31
  2. 2025TabFlex: Scaling Tabular Learning to Millions with Linear Attentioncited by 13
  3. 2025LLM Embeddings for Deep Learning on Tabular Datacited by 11
  4. 2024On the Efficiency of NLP-Inspired Methods for Tabular Deep Learningcited by 6
  5. 2025Understanding the Limits of Deep Tabular Methods with Temporal Shiftcited by 4
  6. 2025State-Space Models for Tabular Prior-Data Fitted Networkscited by 4
  7. 2025Beyond Black-Box Predictions: Identifying Marginal Feature Effects in Tabular Transformer Networkscited by 2
  8. 2025Tabular Data with Class Imbalance: Predicting Electric Vehicle Crash Severity with Pretrained Transformers (TabPFN) and Mamba-Based Modelscited by 2
  9. 2024Mamba for Scalable and Efficient Personalized Recommendationscited by 1
  10. 2026DynaTab: Dynamic Feature Ordering as Neural Rewiring for High-Dimensional Tabular Datacited by 1
  11. 2024APAR: Modeling Irregular Target Functions in Tabular Regression via Arithmetic-Aware Pre-Training and Adaptive-Regularized Fine-Tuningcited by 0
  12. 2025AMF-MedIT: An Efficient Align-Modulation-Fusion Framework for Medical Image-Tabular Datacited by 0
6 more
  1. 2025CALM: A Causal Analysis Language Model for Tabular Data in Complex Systems with Local Scores, Conditional Independence Tests, and Relation Attributescited by 0
  2. 2026From Uniform to Learned Knots: A Study of Spline-Based Numerical Encodings for Tabular Deep Learningcited by 0
  3. 2026An approach with Visual and Tabular Mamba to multimodal medical data using Mixed Fusioncited by 0
  4. 2026iStructTab: Structured Feature Sequencing for Multimodal Learning of Image and Tabular Datacited by 0
  5. 2026TabNSM: Neural Sparse Mixer for Tabular Regressioncited by 0
  6. 2026LoopICL: Looping a single transformer block to solve tabular taskscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024MambaTab: A Plug-and-Play Model for Learning Tabular Data
  2. 2024TabM: Advancing Tabular Deep Learning with Parameter-Efficient Ensembling
  3. 2025Make Still Further Progress: Chain of Thoughts for Tabular Data Leaderboard
  4. 2024A Survey on Deep Tabular Learning
  5. 2023TabR: Tabular Deep Learning Meets Nearest Neighbors in 2023
  6. 2024Optimized Feature Generation for Tabular Data via LLMs with Decision Tree Reasoning
  7. 2024A Data-Centric Perspective on Evaluating Machine Learning Models for Tabular Data
  8. 2025Strengthening LLMs for Tabular Prediction with Structural Priors