# Adaptive Group Lasso Neural Network Models for Functions of Few Variables and Time-Dependent Data

arXiv: https://arxiv.org/abs/2108.10825 · HTML: https://mlsanity.com/paper/2108.10825/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 52 references that are in the map.
- First posted on arXiv 24 Aug 2021.

## Builds on (most cited first, 21 of 21)

- [Benefits of depth in neural networks](https://mlsanity.com/paper/1602.04485/) (2016)
- [Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1708.00588/) (2017)
- [The Power of Depth for Feedforward Neural Networks](https://mlsanity.com/paper/1512.03965/) (2015)
- [Why and When Can Deep -- but Not Shallow -- Networks Avoid the Curse of Dimensionality: a Review](https://mlsanity.com/paper/1611.00740/) (2016)
- [On the Expressive Power of Deep Learning: A Tensor Analysis](https://mlsanity.com/paper/1509.05009/) (2015)
- [Data Driven Governing Equations Approximation Using Deep Neural Networks](https://mlsanity.com/paper/1811.05537/) (2018)
- [Group Sparse Regularization for Deep Neural Networks](https://mlsanity.com/paper/1607.00485/) (2016)
- [Why Deep Neural Networks for Function Approximation?](https://mlsanity.com/paper/1610.04161/) (2016)
- [Extracting Sparse High-Dimensional Dynamics from Limited Data](https://mlsanity.com/paper/1707.08528/) (2017)
- [Nonparametric inference of interaction laws in systems of agents from trajectory data](https://mlsanity.com/paper/1812.06003/) (2018)
- [NeuPDE: Neural Network Based Ordinary and Partial Differential Equations for Modeling Time-Dependent Data](https://mlsanity.com/paper/1908.03190/) (2019)
- [Neural graphical modelling in continuous-time: consistency guarantees and algorithms](https://mlsanity.com/paper/2105.02522/) (2021)
- [Consistent Feature Selection for Analytic Deep Neural Networks](https://mlsanity.com/paper/2010.08097/) (2020)
- [A Bregman Learning Framework for Sparse Neural Networks](https://mlsanity.com/paper/2105.04319/) (2021)
- [Auto-Sizing Neural Networks: With Applications to n-gram Language Models](https://mlsanity.com/paper/1508.05051/) (2015)
- [Auto-Sizing the Transformer Network: Improving Speed, Efficiency, and Performance for Low-Resource Machine Translation](https://mlsanity.com/paper/1910.06717/) (2019)
- [Consistent feature selection for neural networks via Adaptive Group Lasso](https://mlsanity.com/paper/2006.00334/) (2020)
- [A General Family of Stochastic Proximal Gradient Methods for Deep Learning](https://mlsanity.com/paper/2007.07484/) (2020)
- [Generalization and Robustness of Batched Weighted Average Algorithm with V-geometrically Ergodic Markov Data](https://mlsanity.com/paper/1406.3166/) (2014)
- [Learning From Non-iid Data: Fast Rates for the One-vs-All Multiclass Plug-in Classifiers](https://mlsanity.com/paper/1408.2714/) (2014)
- [Methods to Recover Unknown Processes in Partial Differential Equations Using Data](https://mlsanity.com/paper/2003.02387/) (2020)

## Built on by (most cited first, 2 of 2)

- [A Generalization Bound of Deep Neural Networks for Dependent Data](https://mlsanity.com/paper/2310.05892/) (2023)
- [AutoNFS: Automatic Neural Feature Selection](https://mlsanity.com/paper/2503.13304/) (2025)

## Abstract (arXiv)

In this paper, we propose an adaptive group Lasso deep neural network for high-dimensional function approximation where input data are generated from a dynamical system and the target function depends on few active variables or few linear combinations of variables. We approximate the target function by a deep neural network and enforce an adaptive group Lasso constraint to the weights of a suitable hidden layer in order to represent the constraint on the target function. We utilize the proximal algorithm to optimize the penalized loss function. Using the non-negative property of the Bregman distance, we prove that the proposed optimization procedure achieves loss decay. Our empirical studies show that the proposed method outperforms recent state-of-the-art methods including the sparse dictionary matrix method, neural networks with or without group Lasso penalty.