mlsanityData to 7 Oct 2026

DynaTab: Dynamic Feature Ordering as Neural Rewiring for High-Dimensional Tabular Data

arXiv 2605.03430 ↗First posted 5 May 2026cs.LG cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

High-dimensional tabular data lacks a natural feature order, limiting the applicability of permutation-sensitive deep learning models. We propose DynaTab, a dynamic feature ordering-enabled architecture inspired by neural rewiring. We introduce a lightweight criterion that predicts when feature permutation will benefit a dataset by quantifying its intrinsic complexity. DynaTab dynamically reorders features via a neural rewiring algorithm and processes them through a compact, dynamic order-aware combination of separate learned positional embedding, importance-based gating, and masked attention layers, compatible with any sequence-sensitive backbone. Trained end-to-end with bespoke dynamic feature ordering (DFO) and dispersion losses, DynaTab achieves statistically significant gains, particularly on high-dimensional datasets, where it is benchmarked against 45 state-of-the-art baselines across 36 different real-world tabular datasets. Our results position DynaTab as a compelling new paradigm for high-dimensional tabular deep learning.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 45 of its 105 references that are in the map. First posted on arXiv 5 May 2026.

Builds on

45 of 105 references.

References · 45

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  4. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  5. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
  6. 2017Deep Setscited by 1,394
  7. 2018Self-Attention with Relative Position Representationscited by 1,112
  8. 2015Pointer Networkscited by 882
  9. 2021Revisiting Deep Learning Models for Tabular Datacited by 725
  10. 2017DeepFM: A Factorization-Machine based Neural Network for CTR Predictioncited by 673
  11. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 643
  12. 2019TabNet: Attentive Interpretable Tabular Learningcited by 640
33 more
  1. 2021Tabular Data: Deep Learning is Not All You Needcited by 530
  2. 2020TabTransformer: Tabular Data Modeling Using Contextual Embeddingscited by 398
  3. 2017Deep & Cross Network for Ad Click Predictionscited by 392
  4. 2022TabDDPM: Modelling Tabular Data with Diffusion Modelscited by 373
  5. 2018AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networkscited by 319
  6. 2021SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Trainingcited by 310
  7. 2025TabICL: A Tabular Foundation Model for In-Context Learning on Large Datacited by 267
  8. 2018Learning to Explain: An Information-Theoretic Perspective on Model Interpretationcited by 248
  9. 2019Neural Oblivious Decision Ensembles for Deep Learning on Tabular Datacited by 200
  10. 2022On Embeddings for Numerical Features in Tabular Deep Learningcited by 185
  11. 2024TabM: Advancing Tabular Deep Learning with Parameter-Efficient Ensemblingcited by 105
  12. 2023TabR: Tabular Deep Learning Meets Nearest Neighbors in 2023cited by 73
  13. 2021Have We Learned to Explain?: How Interpretability Methods Can Learn to Encode Predictions in their Interpretationscited by 59
  14. 2020Pointer Graph Networkscited by 52
  15. 2021DANets: Deep Abstract Networks for Tabular Data Classification and Regressioncited by 50
  16. 2022T2G-Former: Organizing Tabular Features into Relation Graphs Promotes Heterogeneous Feature Interactioncited by 41
  17. 2023Trompt: Towards a Better Deep Neural Network for Tabular Datacited by 36
  18. 2023HYTREL: Hypergraph-enhanced Tabular Data Representation Learningcited by 35
  19. 2021Locally Sparse Neural Networks for Tabular Biomedical Datacited by 34
  20. 2023TANGOS: Regularizing Tabular Neural Networks through Gradient Orthogonalization and Specializationcited by 32
  21. 2025Turning Tabular Foundation Models into Graph Foundation Modelscited by 30
  22. 2024TabReD: Analyzing Pitfalls and Filling the Gaps in Tabular Deep Learning Benchmarkscited by 29
  23. 2021Graph Pointer Neural Networkscited by 27
  24. 2019Gated Linear Networkscited by 26
  25. 2023Brain-inspired learning in artificial neural networks: a reviewcited by 26
  26. 2021PyTorch Tabular: A Framework for Deep Learning with Tabular Datacited by 20
  27. 2025The Order Effect: Investigating Prompt Sensitivity to Input Order in LLMscited by 19
  28. 2023ProtoGate: Prototype-based Neural Networks with Global-to-local Feature Selection for Tabular Biomedical Datacited by 18
  29. 2024Mambular: A Sequential Model for Tabular Deep Learningcited by 18
  30. 2024MambaTab: A Plug-and-Play Model for Learning Tabular Datacited by 14
  31. 2024On the Efficiency of NLP-Inspired Methods for Tabular Deep Learningcited by 6
  32. 2024TabSeq: A Framework for Deep Learning on Tabular Data via Sequential Orderingcited by 5
  33. 2020Neural Networks, Artificial Intelligence and the Computational Braincited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026BSTabDiff: Block-Subunit Diffusion Priors for High-Dimensional Tabular Data Generationcited by 0