# Factor Augmented Sparse Throughput Deep ReLU Neural Networks for High Dimensional Regression

arXiv: https://arxiv.org/abs/2210.02002 · HTML: https://mlsanity.com/paper/2210.02002/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Oct 2022.

## Built on by (most cited first, 30 of 31)

- [Deep Neural Networks for Nonparametric Interaction Models with Diverging Dimension](https://mlsanity.com/paper/2302.05851/) (2023)
- [Tensor-view Topological Graph Neural Network](https://mlsanity.com/paper/2401.12007/) (2024)
- [Neural networks for geospatial data](https://mlsanity.com/paper/2304.09157/) (2023)
- [UTOPIA: Universally Trainable Optimal Prediction Intervals Aggregation](https://mlsanity.com/paper/2306.16549/) (2023)
- [Causality Pursuit from Heterogeneous Environments via Neural Adversarial Invariance Learning](https://mlsanity.com/paper/2405.04715/) (2024)
- [Factor Augmented Tensor-on-Tensor Neural Networks](https://mlsanity.com/paper/2405.19610/) (2024)
- [Deep Transfer Learning: Model Framework and Error Analysis](https://mlsanity.com/paper/2410.09383/) (2024)
- [Representation Retrieval Learning for Heterogeneous Data Integration](https://mlsanity.com/paper/2503.09494/) (2025)
- [Representation-Enhanced Neural Knowledge Integration with Application to Large-Scale Medical Ontology Learning](https://mlsanity.com/paper/2410.07454/) (2024)
- [Inference for Deep Neural Network Estimators in Generalized Nonparametric Models](https://mlsanity.com/paper/2504.09347/) (2025)
- [Optimally taming biases in black-box models for efficient semiparametric estimation](https://mlsanity.com/paper/2606.06368/) (2026)
- [Factor Informed Double Deep Learning For Average Treatment Effect Estimation](https://mlsanity.com/paper/2508.17136/) (2025)
- [A general framework for deep learning](https://mlsanity.com/paper/2512.23425/) (2025)
- [Factor Adjusted Spectral Clustering for Mixture Models](https://mlsanity.com/paper/2408.12564/) (2024)
- [Semi-supervised Clustering Through Representation Learning of Large-scale EHR Data](https://mlsanity.com/paper/2505.20731/) (2025)
- [Supervised Dynamic Dimension Reduction with Deep Neural Network](https://mlsanity.com/paper/2508.03546/) (2025)
- [CINDES: Classification induced neural density estimator and simulator](https://mlsanity.com/paper/2510.00367/) (2025)
- [Optimal use of a black-box learner in semiparametric estimation](https://mlsanity.com/paper/2607.21541/) (2026)
- [Generalized Factor Neural Network Model for High-dimensional Regression](https://mlsanity.com/paper/2502.11310/) (2025)
- [Boosting Statistic Learning with Synthetic Data from Pretrained Large Models](https://mlsanity.com/paper/2505.04992/) (2025)
- [Feature Augmentations for High-Dimensional Learning](https://mlsanity.com/paper/2509.00232/) (2025)
- [NeuroMAS: Multi-Agent Systems as Neural Networks with Joint Reinforcement Learning](https://mlsanity.com/paper/2605.16757/) (2026)
- [Dual-Channel Tensor Neural Networks: Finite-Sample Theory and Conformal Structure Selection](https://mlsanity.com/paper/2605.19122/) (2026)
- [Factor Augmented High-Dimensional SGD](https://mlsanity.com/paper/2605.19291/) (2026)
- [Rate-optimal neural boundary detection from unlabeled noisy images](https://mlsanity.com/paper/2606.00715/) (2026)
- [Mitigating the Curse of Dimensionality in Uniform Convergence of Deep Neural Networks via Smooth Activations](https://mlsanity.com/paper/2606.05599/) (2026)
- [MediEncoder: Nonlinear Representation Learning for High-Dimensional Causal Mediation Analysis](https://mlsanity.com/paper/2606.30648/) (2026)
- [Deep Multitask Learning for Mixed-Type Outcomes with Shared Sparsity](https://mlsanity.com/paper/2607.00995/) (2026)
- [Minimax Theory of Likelihood-Based Deep Learning for Speckle Regression](https://mlsanity.com/paper/2607.14064/) (2026)
- [Unveiling Invariant and Transferable Latent Factors Across Heterogeneous Environments via ATLAS](https://mlsanity.com/paper/2607.18209/) (2026)

## Abstract (arXiv)

This paper introduces a Factor Augmented Sparse Throughput (FAST) model that utilizes both latent factors and sparse idiosyncratic components for nonparametric regression. The FAST model bridges factor models on one end and sparse nonparametric models on the other end. It encompasses structured nonparametric models such as factor augmented additive models and sparse low-dimensional nonparametric interaction models and covers the cases where the covariates do not admit factor structures. Via diversified projections as estimation of latent factor space, we employ truncated deep ReLU networks to nonparametric factor regression without regularization and to a more general FAST model using nonconvex regularization, resulting in factor augmented regression using neural network (FAR-NN) and FAST-NN estimators respectively. We show that FAR-NN and FAST-NN estimators adapt to the unknown low-dimensional structure using hierarchical composition models in nonasymptotic minimax rates. We also study statistical learning for the factor augmented sparse additive model using a more specific neural network architecture. Our results are applicable to the weak dependent cases without factor structures. In proving the main technical result for FAST-NN, we establish a new deep ReLU network approximation result that contributes to the foundation of neural network theory. Our theory and methods are further supported by simulation studies and an application to macroeconomic data.