# Feature Augmentations for High-Dimensional Learning

arXiv: https://arxiv.org/abs/2509.00232 · HTML: https://mlsanity.com/paper/2509.00232/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 48 references that are in the map.
- First posted on arXiv 29 Aug 2025.

## Builds on (most cited first, 11 of 11)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Tabular Data: Deep Learning is Not All You Need](https://mlsanity.com/paper/2106.03253/) (2021)
- [Factor Augmented Sparse Throughput Deep ReLU Neural Networks for High Dimensional Regression](https://mlsanity.com/paper/2210.02002/) (2022)
- [Factor Models for High-Dimensional Tensor Time Series](https://mlsanity.com/paper/1905.07530/) (2019)
- [Estimating Number of Factors by Adjusted Eigenvalues Thresholding](https://mlsanity.com/paper/1909.10710/) (2019)
- [Factor Models for Matrix-Valued High-Dimensional Time Series](https://mlsanity.com/paper/1610.01889/) (2016)
- [Factor-Adjusted Regularized Model Selection](https://mlsanity.com/paper/1612.08490/) (2016)
- [Learning Latent Factors from Diversified Projections and its Applications to Over-Estimated and Weak Factors](https://mlsanity.com/paper/1908.01252/) (2019)
- [Application of Natural Language Processing in Financial Risk Detection](https://mlsanity.com/paper/2406.09765/) (2024)
- [Factor Augmented Matrix Regression](https://mlsanity.com/paper/2405.17744/) (2024)
- [Feature Augmentation via Nonparametrics and Selection (FANS) in High Dimensional Classification](https://mlsanity.com/paper/1401.0211/) (2013)

## Abstract (arXiv)

High-dimensional measurements are often correlated which motivates their approximation by factor models. This holds also true when features are engineered via low-dimensional interactions or kernel tricks. This often results in over parametrization and requires a fast dimensionality reduction. We propose a simple technique to enhance the performance of supervised learning algorithms by augmenting features with factors extracted from design matrices and their transformations. This is implemented by using the factors and idiosyncratic residuals which significantly weaken the correlations between input variables and hence increase the interpretability of learning algorithms and numerical stability. Extensive experiments on various algorithms and real-world data in diverse fields are carried out, among which we put special emphasis on the stock return prediction problem with Chinese financial news data due to the increasing interest in NLP problems in financial studies. We verify the capability of the proposed feature augmentation approach to boost overall prediction performance with the same algorithm. The approach bridges a gap in research that has been overlooked in previous studies, which focus either on collecting additional data or constructing more powerful algorithms, whereas our method lies in between these two directions using a simple PCA augmentation.