# Neural Network Characterization and Entropy Regulated Data Balancing through Principal Component Analysis

arXiv: https://arxiv.org/abs/2312.01392 · HTML: https://mlsanity.com/paper/2312.01392/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 115 references that are in the map.
- First posted on arXiv 3 Dec 2023.

## Builds on (most cited first, 18 of 18)

- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [A systematic study of the class imbalance problem in convolutional neural networks](https://mlsanity.com/paper/1710.05381/) (2017)
- [Tutorial on Variational Autoencoders](https://mlsanity.com/paper/1606.05908/) (2016)
- [Cost Sensitive Learning of Deep Feature Representations from Imbalanced Data](https://mlsanity.com/paper/1508.03422/) (2015)
- [Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis](https://mlsanity.com/paper/1601.04589/) (2016)
- [Variational Autoencoders Pursue PCA Directions (by Accident)](https://mlsanity.com/paper/1812.06775/) (2018)
- [PCANet: A Simple Deep Learning Baseline for Image Classification?](https://mlsanity.com/paper/1404.3606/) (2014)
- [Machine Learning for the Geosciences: Challenges and Opportunities](https://mlsanity.com/paper/1711.04708/) (2017)
- [Empirical comparison between autoencoders and traditional dimensionality reduction methods](https://mlsanity.com/paper/2103.04874/) (2021)
- [Variational Laplace Autoencoders](https://mlsanity.com/paper/2211.17267/) (2022)
- [High-Dimensional Smoothed Entropy Estimation via Dimensionality Reduction](https://mlsanity.com/paper/2305.04712/) (2023)
- [Variational Autoencoder Analysis of Ising Model Statistical Distributions and Phase Transitions](https://mlsanity.com/paper/2104.06368/) (2021)
- [Predicting Student Dropout in Higher Education](https://mlsanity.com/paper/1606.06364/) (2016)
- [Entropy Approximation by Machine Learning Regression: Application for Irregularity Evaluation of Images in Remote Sensing](https://mlsanity.com/paper/2210.06901/) (2022)
- [Geometric SMOTE: Effective oversampling for imbalanced learning through a geometric extension of SMOTE](https://mlsanity.com/paper/1709.07377/) (2017)
- [Rotated Digit Recognition by Variational Autoencoders with Fixed Output Distributions](https://mlsanity.com/paper/2206.13388/) (2022)
- [Data Dimension Reduction makes ML Algorithms efficient](https://mlsanity.com/paper/2211.09392/) (2022)
- [Deep Feature Learning for Wireless Spectrum Data](https://mlsanity.com/paper/2308.03530/) (2023)

## Built on by (most cited first, 1 of 1)

- [Nonlinearity Enhanced Adaptive Activation Functions](https://mlsanity.com/paper/2403.19896/) (2024)

## Abstract (arXiv)

This paper examines in detail the geometric structure of principal component analysis (PCA) by considering in detail the distributions of both unrotated and rotated MNIST digits in the space defined by the lowest order PCA components. Since digits possessing salient geometric features are mapped to restricted regions far from the origin, they are predicted by neural networks with a greater accuracy than digits that are mapped to broad, diffuse and overlapping volumes of the low order PCA space. Motivated by these results, a new quantity, the local PCA entropy, obtained by dividing the spatial region spanned by the low order principal components into histogram bins and evaluating the entropy associated with the number of occurrences of each input class within a bin, is introduced. The metric locates the input data records that yield the largest confusion in prediction accuracy within reduced coordinate volumes that optimally discriminate among geometric features. As an example of the potential utility of the local PCA entropy, a simple data balancing procedure is realized by oversampling the data records in regions of large local entropy.