# Reducing Data Complexity using Autoencoders with Class-informed Loss Functions

arXiv: https://arxiv.org/abs/2111.06142 · HTML: https://mlsanity.com/paper/2111.06142/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 47 references that are in the map.
- First posted on arXiv 11 Nov 2021.

## Builds on (most cited first, 9 of 9)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [How Complex is your classification problem? A survey on measuring classification complexity](https://mlsanity.com/paper/1808.03591/) (2018)
- [A practical tutorial on autoencoders for nonlinear feature fusion: Taxonomy, models, software and guidelines](https://mlsanity.com/paper/1801.01586/) (2018)
- [An analysis on the use of autoencoders for representation learning: fundamentals, learning task case studies, explainability and challenges](https://mlsanity.com/paper/2005.10516/) (2020)
- [Revisiting Data Complexity Metrics Based on Morphology for Overlap and Imbalance: Snapshot, New Overlap Number of Balls Metrics and Singular Problems Prospect](https://mlsanity.com/paper/2007.07935/) (2020)
- [Classification of Imbalanced Data with a Geometric Digraph Family](https://mlsanity.com/paper/1904.04564/) (2019)
- [A Tutorial on Distance Metric Learning: Mathematical Foundations, Algorithms, Experimental Analysis, Prospects and Challenges (with Appendices on Mathematical Background and Detailed Algorithms Explanation)](https://mlsanity.com/paper/1812.05944/) (2018)

## Built on by (most cited first, 3 of 3)

- [MoE-Enhanced Explainable Deep Manifold Transformation for Complex Data Embedding and Visualization](https://mlsanity.com/paper/2410.19504/) (2024)
- [Are We Using Autoencoders in a Wrong Way?](https://mlsanity.com/paper/2309.01532/) (2023)
- [A classification performance evaluation measure considering data separability](https://mlsanity.com/paper/2211.05433/) (2022)

## Abstract (arXiv)

Available data in machine learning applications is becoming increasingly complex, due to higher dimensionality and difficult classes. There exists a wide variety of approaches to measuring complexity of labeled data, according to class overlap, separability or boundary shapes, as well as group morphology. Many techniques can transform the data in order to find better features, but few focus on specifically reducing data complexity. Most data transformation methods mainly treat the dimensionality aspect, leaving aside the available information within class labels which can be useful when classes are somehow complex. This paper proposes an autoencoder-based approach to complexity reduction, using class labels in order to inform the loss function about the adequacy of the generated variables. This leads to three different new feature learners, Scorer, Skaler and Slicer. They are based on Fisher's discriminant ratio, the Kullback-Leibler divergence and least-squares support vector machines, respectively. They can be applied as a preprocessing stage for a binary classification problem. A thorough experimentation across a collection of 27 datasets and a range of complexity and classification metrics shows that class-informed autoencoders perform better than 4 other popular unsupervised feature extraction techniques, especially when the final objective is using the data for a classification task.