# TabMixer: Excavating Label Distribution Learning with Small-scale Features

arXiv: https://arxiv.org/abs/2210.13852 · HTML: https://mlsanity.com/paper/2210.13852/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 24 references that are in the map.
- First posted on arXiv 25 Oct 2022.

## Builds on (most cited first, 9 of 9)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [TabNet: Attentive Interpretable Tabular Learning](https://mlsanity.com/paper/1908.07442/) (2019)
- [TransTab: Learning Transferable Tabular Transformers Across Tables](https://mlsanity.com/paper/2205.09328/) (2022)
- [Label Distribution Learning](https://mlsanity.com/paper/1408.6027/) (2014)
- [Uncertainty-aware Label Distribution Learning for Facial Expression Recognition](https://mlsanity.com/paper/2209.10448/) (2022)
- [Towards Speaker Age Estimation with Label Distribution Learning](https://mlsanity.com/paper/2202.11424/) (2022)
- [Label Distribution Learning via Implicit Distribution Representation](https://mlsanity.com/paper/2209.13824/) (2022)

## Abstract (arXiv)

Label distribution learning (LDL) differs from multi-label learning which aims at representing the polysemy of instances by transforming single-label values into descriptive degrees. Unfortunately, the feature space of the label distribution dataset is affected by human factors and the inductive bias of the feature extractor causing uncertainty in the feature space. Especially, for datasets with small-scale feature spaces (the feature space dimension ≈ the label space), the existing LDL algorithms do not perform well. To address this issue, we seek to model the uncertainty augmentation of the feature space to alleviate the problem in LDL tasks. Specifically, we start with augmenting each feature value in the feature vector of a sample into a vector (sampling on a Gaussian distribution function). Which, the variance parameter of the Gaussian distribution function is learned by using a sub-network, and the mean parameter is filled by this feature value. Then, each feature vector is augmented to a matrix which is fed into a mixer with local attention (textitTabMixer) to extract the latent feature. Finally, the latent feature is squeezed to yield an accurate label distribution via a squeezed network. Extensive experiments verify that our proposed algorithm can be competitive compared to other LDL algorithms on several benchmarks.