# ReConTab: Regularized Contrastive Representation Learning for Tabular Data

arXiv: https://arxiv.org/abs/2310.18541 · HTML: https://mlsanity.com/paper/2310.18541/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 83 references that are in the map.
- First posted on arXiv 28 Oct 2023.

## Builds on (most cited first, 30 of 37)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [TabNet: Attentive Interpretable Tabular Learning](https://mlsanity.com/paper/1908.07442/) (2019)
- [Self-Normalizing Neural Networks](https://mlsanity.com/paper/1706.02515/) (2017)
- [Deep Neural Networks and Tabular Data: A Survey](https://mlsanity.com/paper/2110.01889/) (2021)
- [Masked Autoencoders As Spatiotemporal Learners](https://mlsanity.com/paper/2205.09113/) (2022)
- [TabTransformer: Tabular Data Modeling Using Contextual Embeddings](https://mlsanity.com/paper/2012.06678/) (2020)
- [TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data](https://mlsanity.com/paper/2005.08314/) (2020)
- [DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems](https://mlsanity.com/paper/2008.13535/) (2020)
- [AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks](https://mlsanity.com/paper/1810.11921/) (2018)
- [SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training](https://mlsanity.com/paper/2106.01342/) (2021)
- [Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learning](https://mlsanity.com/paper/2212.14532/) (2022)
- [Revisiting Self-Supervised Visual Representation Learning](https://mlsanity.com/paper/1901.09005/) (2019)
- [Neural Oblivious Decision Ensembles for Deep Learning on Tabular Data](https://mlsanity.com/paper/1909.06312/) (2019)
- [SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption](https://mlsanity.com/paper/2106.15147/) (2021)
- [Self-Supervised Representation Learning: Introduction, Advances and Challenges](https://mlsanity.com/paper/2110.09327/) (2021)
- [Graph Adversarial Training: Dynamically Regularizing Based on Graph Structure](https://mlsanity.com/paper/1902.08226/) (2019)
- [Secure and Robust Machine Learning for Healthcare: A Survey](https://mlsanity.com/paper/2001.08103/) (2020)
- [Gradient Boosting Neural Networks: GrowNet](https://mlsanity.com/paper/2002.07971/) (2020)

## Built on by (most cited first, 25 of 25)

- [SwitchTab: Switched Autoencoders Are Effective Tabular Learners](https://mlsanity.com/paper/2401.02013/) (2024)
- [Representation Learning for Tabular Data: A Comprehensive Survey](https://mlsanity.com/paper/2504.16109/) (2025)
- [MaxK-GNN: Extremely Fast GPU Kernel Design for Accelerating Graph Neural Networks Training](https://mlsanity.com/paper/2312.08656/) (2023)
- [Residual-based Language Models are Free Boosters for Biomedical Imaging](https://mlsanity.com/paper/2403.17343/) (2024)
- [Adaptive Ensembles of Fine-Tuned Transformers for LLM-Generated Text Detection](https://mlsanity.com/paper/2403.13335/) (2024)
- [A Survey on Deep Tabular Learning](https://mlsanity.com/paper/2410.12034/) (2024)
- [A Survey on Self-Supervised Learning for Non-Sequential Tabular Data](https://mlsanity.com/paper/2402.01204/) (2024)
- [Rumor Detection with a novel graph neural network approach](https://mlsanity.com/paper/2403.16206/) (2024)
- [The New Agronomists: Language Models are Experts in Crop Management](https://mlsanity.com/paper/2403.19839/) (2024)
- [Image Captioning in news report scenario](https://mlsanity.com/paper/2403.16209/) (2024)
- [In-Trajectory Inverse Reinforcement Learning: Learn Incrementally Before An Ongoing Trajectory Terminates](https://mlsanity.com/paper/2410.15612/) (2024)
- [TabSeq: A Framework for Deep Learning on Tabular Data via Sequential Ordering](https://mlsanity.com/paper/2410.13203/) (2024)
- [Latte: Transfering LLMs` Latent-level Knowledge for Few-shot Tabular Learning](https://mlsanity.com/paper/2505.05237/) (2025)
- [Tree-Regularized Tabular Embeddings](https://mlsanity.com/paper/2403.00963/) (2024)
- [Optimized Coordination Strategy for Multi-Aerospace Systems in Pick-and-Place Tasks By Deep Neural Network](https://mlsanity.com/paper/2412.09877/) (2024)
- [TabDeco: A Comprehensive Contrastive Framework for Decoupled Representations in Tabular Data](https://mlsanity.com/paper/2411.11148/) (2024)
- [MIRRAMS: Learning Robust Tabular Models under Unseen Missingness Shifts](https://mlsanity.com/paper/2507.08280/) (2025)
- [Deep Representation Learning for Multi-functional Degradation Modeling of Community-dwelling Aging Population](https://mlsanity.com/paper/2404.05613/) (2024)
- [TransTARec: Time-Adaptive Translating Embedding Model for Next POI Recommendation](https://mlsanity.com/paper/2404.07096/) (2024)
- [Bridge: A Unified Framework to Knowledge Graph Completion via Language Models and Knowledge Representation](https://mlsanity.com/paper/2411.06660/) (2024)
- [Table2Image: Lightweight Tabular Learning with Generated Proxy Representations and Reliability Diagnostics](https://mlsanity.com/paper/2412.06265/) (2024)
- [Harnessing LLMs Explanations to Boost Surrogate Models in Tabular Data Classification](https://mlsanity.com/paper/2505.05744/) (2025)
- [Random at First, Fast at Last: NTK-Guided Fourier Pre-Processing for Tabular DL](https://mlsanity.com/paper/2506.02406/) (2025)
- [FairContrast: Enhancing Fairness through Contrastive learning and Customized Augmenting Methods on Tabular Data](https://mlsanity.com/paper/2510.02017/) (2025)
- [CAST: Achieving Stable LLM-based Text Analysis for Data Analytics](https://mlsanity.com/paper/2602.15861/) (2026)

## Abstract (arXiv)

Representation learning stands as one of the critical machine learning techniques across various domains. Through the acquisition of high-quality features, pre-trained embeddings significantly reduce input space redundancy, benefiting downstream pattern recognition tasks such as classification, regression, or detection. Nonetheless, in the domain of tabular data, feature engineering and selection still heavily rely on manual intervention, leading to time-consuming processes and necessitating domain expertise. In response to this challenge, we introduce ReConTab, a deep automatic representation learning framework with regularized contrastive learning. Agnostic to any type of modeling task, ReConTab constructs an asymmetric autoencoder based on the same raw features from model inputs, producing low-dimensional representative embeddings. Specifically, regularization techniques are applied for raw feature selection. Meanwhile, ReConTab leverages contrastive learning to distill the most pertinent information for downstream tasks. Experiments conducted on extensive real-world datasets substantiate the framework's capacity to yield substantial and robust performance improvements. Furthermore, we empirically demonstrate that pre-trained embeddings can seamlessly integrate as easily adaptable features, enhancing the performance of various traditional methods such as XGBoost and Random Forest.