# Automatic Node Selection for Deep Neural Networks using Group Lasso Regularization

arXiv: https://arxiv.org/abs/1611.05527 · HTML: https://mlsanity.com/paper/1611.05527/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Nov 2016.

## Built on by (most cited first, 7 of 7)

- [Deep neural networks with dependent weights: Gaussian Process mixture limit, heavy tails, sparsity and compressibility](https://mlsanity.com/paper/2205.08187/) (2022)
- [Spike-and-slab shrinkage priors for structurally sparse Bayesian neural networks](https://mlsanity.com/paper/2308.09104/) (2023)
- [Layer Collapse Can be Induced by Unstructured Pruning](https://mlsanity.com/paper/2404.16890/) (2024)
- [Structured Directional Pruning via Perturbation Orthogonal Projection](https://mlsanity.com/paper/2107.05328/) (2021)
- [A least distance estimator for a multivariate regression model using deep neural networks](https://mlsanity.com/paper/2401.03123/) (2024)
- [Graph Neural Machine: A New Model for Learning with Tabular Data](https://mlsanity.com/paper/2402.02862/) (2024)
- [Structured Output Regularization: a framework for few-shot transfer learning](https://mlsanity.com/paper/2510.08728/) (2025)

## Abstract (arXiv)

We examine the effect of the Group Lasso (gLasso) regularizer in selecting the salient nodes of Deep Neural Network (DNN) hidden layers by applying a DNN-HMM hybrid speech recognizer to TED Talks speech data. We test two types of gLasso regularization, one for outgoing weight vectors and another for incoming weight vectors, as well as two sizes of DNNs: 2048 hidden layer nodes and 4096 nodes. Furthermore, we compare gLasso and L2 regularizers. Our experiment results demonstrate that our DNN training, in which the gLasso regularizer was embedded, successfully selected the hidden layer nodes that are necessary and sufficient for achieving high classification power.