# Boarding for ISS: Imbalanced Self-Supervised: Discovery of a Scaled Autoencoder for Mixed Tabular Datasets

arXiv: https://arxiv.org/abs/2403.15790 · HTML: https://mlsanity.com/paper/2403.15790/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 56 references that are in the map.
- First posted on arXiv 23 Mar 2024.

## Builds on (most cited first, 25 of 25)

- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss](https://mlsanity.com/paper/1906.07413/) (2019)
- [Modeling Tabular data using Conditional GAN](https://mlsanity.com/paper/1907.00503/) (2019)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [A Survey on Contrastive Self-supervised Learning](https://mlsanity.com/paper/2011.00362/) (2020)
- [Tabular Data: Deep Learning is Not All You Need](https://mlsanity.com/paper/2106.03253/) (2021)
- [A systematic study of the class imbalance problem in convolutional neural networks](https://mlsanity.com/paper/1710.05381/) (2017)
- [Deep Neural Networks and Tabular Data: A Survey](https://mlsanity.com/paper/2110.01889/) (2021)
- [Rethinking the Value of Labels for Improving Class-Imbalanced Learning](https://mlsanity.com/paper/2006.07529/) (2020)
- [Mixed-Type Tabular Data Synthesis with Score-based Diffusion in Latent Space](https://mlsanity.com/paper/2310.09656/) (2023)
- [Delving into Deep Imbalanced Regression](https://mlsanity.com/paper/2102.09554/) (2021)
- [Self-supervised Learning is More Robust to Dataset Imbalance](https://mlsanity.com/paper/2110.05025/) (2021)
- [SubTab: Subsetting Features of Tabular Data for Self-Supervised Representation Learning](https://mlsanity.com/paper/2110.04361/) (2021)
- [Balanced MSE for Imbalanced Visual Regression](https://mlsanity.com/paper/2203.16427/) (2022)
- [VAEM: a Deep Generative Model for Heterogeneous Mixed Type Data](https://mlsanity.com/paper/2006.11941/) (2020)
- [RankSim: Ranking Similarity Regularization for Deep Imbalanced Regression](https://mlsanity.com/paper/2205.15236/) (2022)
- [Contrastive Mixup: Self- and Semi-Supervised learning for Tabular Domain](https://mlsanity.com/paper/2108.12296/) (2021)
- [Autoencoding Binary Classifiers for Supervised Anomaly Detection](https://mlsanity.com/paper/1903.10709/) (2019)
- [Robust Variational Autoencoders for Outlier Detection and Repair of Mixed-Type Data](https://mlsanity.com/paper/1907.06671/) (2019)
- [Imbalance-Aware Self-Supervised Learning for 3D Radiomic Representations](https://mlsanity.com/paper/2103.04167/) (2021)
- [Synthetic Dataset Generation of Driver Telematics](https://mlsanity.com/paper/2102.00252/) (2021)
- [Data Augmentation for Imbalanced Regression](https://mlsanity.com/paper/2302.09288/) (2023)
- [Auto-encoder based Model for High-dimensional Imbalanced Industrial Data](https://mlsanity.com/paper/2108.02083/) (2021)
- [Self-Supervised Neural Architecture Search for Imbalanced Datasets](https://mlsanity.com/paper/2109.08580/) (2021)
- [Federated Model Aggregation via Self-Supervised Priors for Highly Imbalanced Medical Image Classification](https://mlsanity.com/paper/2307.14959/) (2023)

## Built on by (most cited first, 2 of 2)

- [Tokenized SAEs: Disentangling SAE Reconstructions](https://mlsanity.com/paper/2502.17332/) (2025)
- [Disentangled Deep Smoothed Bootstrap for Fair Imbalanced Regression](https://mlsanity.com/paper/2508.13829/) (2025)

## Abstract (arXiv)

The field of imbalanced self-supervised learning, especially in the context of tabular data, has not been extensively studied. Existing research has predominantly focused on image datasets. This paper aims to fill this gap by examining the specific challenges posed by data imbalance in self-supervised learning in the domain of tabular data, with a primary focus on autoencoders. Autoencoders are widely employed for learning and constructing a new representation of a dataset, particularly for dimensionality reduction. They are also often used for generative model learning, as seen in variational autoencoders. When dealing with mixed tabular data, qualitative variables are often encoded using a one-hot encoder with a standard loss function (MSE or Cross Entropy). In this paper, we analyze the drawbacks of this approach, especially when categorical variables are imbalanced. We propose a novel metric to balance learning: a Multi-Supervised Balanced MSE. This approach reduces the reconstruction error by balancing the influence of variables. Finally, we empirically demonstrate that this new metric, compared to the standard MSE: i) outperforms when the dataset is imbalanced, especially when the learning process is insufficient, and ii) provides similar results in the opposite case.