mlsanityData to 7 Oct 2026

Recurrent Neural Network Training with Convex Loss and Regularization Functions by Extended Kalman Filtering

arXiv 2111.02673 ↗First posted 4 Nov 2021cs.LG cs.SY eess.SYcited by 8

Not on the map. Why?

Abstract (arXiv)

This paper investigates the use of extended Kalman filtering to train recurrent neural networks with rather general convex loss functions and regularization terms on the network parameters, including ℓ1-regularization. We show that the learning method is competitive with respect to stochastic gradient descent in a nonlinear system identification benchmark and in training a linear system with binary outputs. We also explore the use of the algorithm in data-driven nonlinear model predictive control and its relation with disturbance models for offset-free closed-loop tracking.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 8 of its 42 references that are in the map. First posted on arXiv 4 Nov 2021.

Builds on

8 of 42 references.

References · 8

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2019On the Convergence of Adam and Beyondcited by 834
  4. 2019Model structures and fitting criteria for system identification with neural networkscited by 16
  5. 2019LSTM Neural Networks: Input to State Stability and Probabilistic Safety Verificationcited by 12
  6. 2021Neural Network Training as an Optimal Control Problem: An Augmented Lagrangian Approachcited by 6
  7. 2014Controlling the level of sparsity in MPCcited by 2
  8. 2020Reduction of the Number of Variables in Parametric Constrained Least-Squares Problemscited by 1

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 1; 2023: 2; 2024: 2; 2025: 2.

8 papers.

  1. 2021Training Recurrent Neural Networks by Sequential Least Squares and the Alternating Direction Method of Multiplierscited by 8
  2. 2024An L-BFGS-B approach for linear and nonlinear system identification under ℓ1 and group-Lasso regularizationcited by 8
  3. 2024AI-Aided Kalman Filterscited by 3
  4. 2023On the Dynamics of Learning Time-Aware Behavior with Recurrent Neural Networkscited by 1
  5. 2022An active learning method for solving competitive multi-agent decision-making and control problemscited by 0
  6. 2023Data-Based Design of Multi-Model Inferential Sensorscited by 0
  7. 2025IRNN: Innovation-driven Recurrent Neural Network for Time-Series Data Modeling and Predictioncited by 0
  8. 2025A proximal augmented Lagrangian method for nonconvex optimization with equality and inequality constraintscited by 0