# Deep learning with transfer functions: new applications in system identification

arXiv: https://arxiv.org/abs/2104.09839 · HTML: https://mlsanity.com/paper/2104.09839/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 19 references that are in the map.
- First posted on arXiv 20 Apr 2021.

## Builds on (most cited first, 7 of 7)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Deep Learning in Neural Networks: An Overview](https://mlsanity.com/paper/1404.7828/) (2014)
- [Continuous-time system identification with neural networks: Model structures and fitting criteria](https://mlsanity.com/paper/2006.02915/) (2020)
- [dynoNet: a neural network architecture for learning dynamical systems](https://mlsanity.com/paper/2006.02250/) (2020)
- [Deep Convolutional Networks in System Identification](https://mlsanity.com/paper/1909.01730/) (2019)
- [Model structures and fitting criteria for system identification with neural networks](https://mlsanity.com/paper/1911.13034/) (2019)
- [Parametric identification of parallel Wiener-Hammerstein systems](https://mlsanity.com/paper/1708.06543/) (2017)

## Built on by (most cited first, 1 of 1)

- [Causal Convolutional Neural Networks as Finite Impulse Response Filters](https://mlsanity.com/paper/2510.24125/) (2025)

## Abstract (arXiv)

This paper presents a linear dynamical operator described in terms of a rational transfer function, endowed with a well-defined and efficient back-propagation behavior for automatic derivatives computation. The operator enables end-to-end training of structured networks containing linear transfer functions and other differentiable units {by} exploiting standard deep learning software. Two relevant applications of the operator in system identification are presented. The first one consists in the integration of {prediction error methods} in deep learning. The dynamical operator is included as {the} last layer of a neural network in order to obtain the optimal one-step-ahead prediction error. The second one considers identification of general block-oriented models from quantized data. These block-oriented models are constructed by combining linear dynamical operators with static nonlinearities described as standard feed-forward neural networks. A custom loss function corresponding to the log-likelihood of quantized output observations is defined. For gradient-based optimization, the derivatives of the log-likelihood are computed by applying the back-propagation algorithm through the whole network. Two system identification benchmarks are used to show the effectiveness of the proposed methodologies.