# A novel Deep Neural Network architecture for non-linear system identification

arXiv: https://arxiv.org/abs/2106.03078 · HTML: https://mlsanity.com/paper/2106.03078/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 15 references that are in the map.
- First posted on arXiv 6 Jun 2021.

## Builds on (most cited first, 3 of 3)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)

## Built on by (most cited first, 7 of 7)

- [Multi-Modal Hallucination Control by Visual Information Grounding](https://mlsanity.com/paper/2403.14003/) (2024)
- [Deep networks for system identification: a Survey](https://mlsanity.com/paper/2301.12832/) (2023)
- [B'MOJO: Hybrid State Space Realizations of Foundation Models with Eidetic and Fading Memory](https://mlsanity.com/paper/2407.06324/) (2024)
- [Stacked Residuals of Dynamic Layers for Time Series Anomaly Detection](https://mlsanity.com/paper/2202.12457/) (2022)
- [End-to-End Learning Framework for Solving Non-Markovian Optimal Control](https://mlsanity.com/paper/2502.04649/) (2025)
- [NARX Identification using Derivative-Based Regularized Neural Networks](https://mlsanity.com/paper/2204.05892/) (2022)
- [DeepBayes -- an estimator for parameter estimation in stochastic nonlinear dynamical models](https://mlsanity.com/paper/2205.02264/) (2022)

## Abstract (arXiv)

We present a novel Deep Neural Network (DNN) architecture for non-linear system identification. We foster generalization by constraining DNN representational power. To do so, inspired by fading memory systems, we introduce inductive bias (on the architecture) and regularization (on the loss function). This architecture allows for automatic complexity selection based solely on available data, in this way the number of hyper-parameters that must be chosen by the user is reduced. Exploiting the highly parallelizable DNN framework (based on Stochastic optimization methods) we successfully apply our method to large scale datasets.