# DANets: Deep Abstract Networks for Tabular Data Classification and Regression

arXiv: https://arxiv.org/abs/2112.02962 · HTML: https://mlsanity.com/paper/2112.02962/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 44 references that are in the map.
- First posted on arXiv 6 Dec 2021.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Language Modeling with Gated Convolutional Networks](https://mlsanity.com/paper/1612.08083/) (2016)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [TabNet: Attentive Interpretable Tabular Learning](https://mlsanity.com/paper/1908.07442/) (2019)
- [RepVGG: Making VGG-style ConvNets Great Again](https://mlsanity.com/paper/2101.03697/) (2021)
- [Train longer, generalize better: closing the generalization gap in large batch training of neural networks](https://mlsanity.com/paper/1705.08741/) (2017)
- [Neural Oblivious Decision Ensembles for Deep Learning on Tabular Data](https://mlsanity.com/paper/1909.06312/) (2019)
- [Sparse Sequence-to-Sequence Models](https://mlsanity.com/paper/1905.05702/) (2019)
- [Deep Neural Decision Trees](https://mlsanity.com/paper/1806.06988/) (2018)
- [Quasi-hyperbolic momentum and Adam for deep learning](https://mlsanity.com/paper/1810.06801/) (2018)
- [E.T.-RNN: Applying Deep Learning to Credit Loan Applications](https://mlsanity.com/paper/1911.02496/) (2019)
- [DNF-Net: A Neural Architecture for Tabular Data](https://mlsanity.com/paper/2006.06465/) (2020)
- [Multi-Layered Gradient Boosting Decision Trees](https://mlsanity.com/paper/1806.00007/) (2018)
- [Benchmarking and Optimization of Gradient Boosting Decision Tree Algorithms](https://mlsanity.com/paper/1809.04559/) (2018)
- [Random Hinge Forest for Differentiable Learning](https://mlsanity.com/paper/1802.03882/) (2018)

## Built on by (most cited first, 30 of 50)

- [When Do Neural Nets Outperform Boosted Trees on Tabular Data?](https://mlsanity.com/paper/2305.02997/) (2023)
- [TransTab: Learning Transferable Tabular Transformers Across Tables](https://mlsanity.com/paper/2205.09328/) (2022)
- [Making Pre-trained Language Models Great on Tabular Prediction](https://mlsanity.com/paper/2403.01841/) (2024)
- [TALENT: A Tabular Analytics and Learning Toolbox](https://mlsanity.com/paper/2407.04057/) (2024)
- [TuneTables: Context Optimization for Scalable Prior-Data Fitted Networks](https://mlsanity.com/paper/2402.11137/) (2024)
- [T2G-Former: Organizing Tabular Features into Relation Graphs Promotes Heterogeneous Feature Interaction](https://mlsanity.com/paper/2211.16887/) (2022)
- [A Closer Look at TabPFN v2: Understanding Its Strengths and Extending Its Capabilities](https://mlsanity.com/paper/2502.17361/) (2025)
- [Representation Learning for Tabular Data: A Comprehensive Survey](https://mlsanity.com/paper/2504.16109/) (2025)
- [ExcelFormer: A neural network surpassing GBDTs on tabular data](https://mlsanity.com/paper/2301.02819/) (2023)
- [Mixture of In-Context Prompters for Tabular PFNs](https://mlsanity.com/paper/2405.16156/) (2024)
- [Revisiting Nearest Neighbor for Tabular Data: A Deep Tabular Baseline Two Decades Later](https://mlsanity.com/paper/2407.03257/) (2024)
- [PTaRL: Prototype-based Tabular Representation Learning via Space Calibration](https://mlsanity.com/paper/2407.05364/) (2024)
- [Towards Cross-Table Masked Pretraining for Web Data Mining](https://mlsanity.com/paper/2307.04308/) (2023)
- [Language Modeling on Tabular Data: A Survey of Foundations, Techniques and Evolution](https://mlsanity.com/paper/2408.10548/) (2024)
- [A Survey on Deep Tabular Learning](https://mlsanity.com/paper/2410.12034/) (2024)
- [A Data-Centric Perspective on Evaluating Machine Learning Models for Tabular Data](https://mlsanity.com/paper/2407.02112/) (2024)
- [TabFlex: Scaling Tabular Learning to Millions with Linear Attention](https://mlsanity.com/paper/2506.05584/) (2025)
- [GANDALF: Gated Adaptive Network for Deep Automated Learning of Features](https://mlsanity.com/paper/2207.08548/) (2022)
- [Unlocking the Transferability of Tokens in Deep Models for Tabular Data](https://mlsanity.com/paper/2310.15149/) (2023)
- [NCART: Neural Classification and Regression Tree for Tabular Data](https://mlsanity.com/paper/2307.12198/) (2023)
- [DIWIFT: Discovering Instance-wise Influential Features for Tabular Data](https://mlsanity.com/paper/2207.02773/) (2022)
- [TrialBench: Multi-Modal Artificial Intelligence-Ready Clinical Trial Datasets](https://mlsanity.com/paper/2407.00631/) (2024)
- [Small Models are LLM Knowledge Triggers on Medical Tabular Prediction](https://mlsanity.com/paper/2403.01570/) (2024)
- [Retrieval-aligned Tabular Foundation Models Enable Robust Clinical Risk Prediction in Electronic Health Records Under Real-world Constraints](https://mlsanity.com/paper/2604.01841/) (2026)
- [Improve Deep Forest with Learnable Layerwise Augmentation Policy Schedule](https://mlsanity.com/paper/2309.09030/) (2023)
- [RDBench: ML Benchmark for Relational Databases](https://mlsanity.com/paper/2310.16837/) (2023)
- [No One-Size-Fits-All Neurons: Task-based Neurons for Artificial Neural Networks](https://mlsanity.com/paper/2405.02369/) (2024)
- [Team up GBDTs and DNNs: Advancing Efficient and Effective Tabular Prediction with Tree-hybrid MLPs](https://mlsanity.com/paper/2407.09790/) (2024)
- [Make Still Further Progress: Chain of Thoughts for Tabular Data Leaderboard](https://mlsanity.com/paper/2505.13421/) (2025)
- [ProtoNAM: Prototypical Neural Additive Models for Interpretable Deep Tabular Learning](https://mlsanity.com/paper/2410.04723/) (2024)

## Abstract (arXiv)

Tabular data are ubiquitous in real world applications. Although many commonly-used neural components (e.g., convolution) and extensible neural networks (e.g., ResNet) have been developed by the machine learning community, few of them were effective for tabular data and few designs were adequately tailored for tabular data structures. In this paper, we propose a novel and flexible neural component for tabular data, called Abstract Layer (AbstLay), which learns to explicitly group correlative input features and generate higher-level features for semantics abstraction. Also, we design a structure re-parameterization method to compress the learned AbstLay, thus reducing the computational complexity by a clear margin in the reference phase. A special basic block is built using AbstLays, and we construct a family of Deep Abstract Networks (DANets) for tabular data classification and regression by stacking such blocks. In DANets, a special shortcut path is introduced to fetch information from raw tabular features, assisting feature interactions across different levels. Comprehensive experiments on seven real-world tabular datasets show that our AbstLay and DANets are effective for tabular data classification and regression, and the computational complexity is superior to competitive methods. Besides, we evaluate the performance gains of DANet as it goes deep, verifying the extendibility of our method. Our code is available at https://github.com/WhatAShot/DANet.