# A Note on Dimensionality Reduction in Deep Neural Networks using Empirical Interpolation Method

arXiv: https://arxiv.org/abs/2305.09842 · HTML: https://mlsanity.com/paper/2305.09842/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 22 references that are in the map.
- First posted on arXiv 16 May 2023.

## Builds on (most cited first, 9 of 9)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Deep Neural Networks Motivated by Partial Differential Equations](https://mlsanity.com/paper/1804.04272/) (2018)
- [Robust Training and Initialization of Deep Neural Networks: An Adaptive Basis Viewpoint](https://mlsanity.com/paper/1912.04862/) (2019)
- [Practical sketching algorithms for low-rank matrix approximation](https://mlsanity.com/paper/1609.00048/) (2016)
- [A Deep Learning approach to Reduced Order Modelling of Parameter Dependent Partial Differential Equations](https://mlsanity.com/paper/2103.06183/) (2021)
- [Novel Deep neural networks for solving Bayesian statistical inverse](https://mlsanity.com/paper/2102.03974/) (2021)
- [Novel DNNs for Stiff ODEs with Applications to Chemically Reacting Flows](https://mlsanity.com/paper/2104.01914/) (2021)
- [Deep Neural Nets with Fixed Bias Configuration](https://mlsanity.com/paper/2107.01308/) (2021)
- [A Note on QR-Based Model Reduction: Algorithm, Software, and Gravitational Wave Applications](https://mlsanity.com/paper/1805.06124/) (2018)

## Built on by (most cited first, 1 of 1)

- [Neural empirical interpolation method for nonlinear model reduction](https://mlsanity.com/paper/2406.03562/) (2024)

## Abstract (arXiv)

Empirical interpolation method (EIM) is a well-known technique to efficiently approximate parameterized functions. This paper proposes to use EIM algorithm to efficiently reduce the dimension of the training data within supervised machine learning. This is termed as DNN-EIM. Applications in data science (e.g., MNIST) and parameterized (and time-dependent) partial differential equations (PDEs) are considered. The proposed DNNs in case of classification are trained in parallel for each class. This approach is sequential, i.e., new classes can be added without having to retrain the network. In case of PDEs, a DNN is designed corresponding to each EIM point. Again, these networks can be trained in parallel, for each EIM point. In all cases, the parallel networks require fewer than ten times the number of training weights. Significant gains are observed in terms of training times, without sacrificing accuracy.