# Efficient Time-Series Approximation with Linear Recurrent Neural Networks: Architecture Learning and Predictive Power

arXiv: https://arxiv.org/abs/1802.03308 · HTML: https://mlsanity.com/paper/1802.03308/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Feb 2018.

## Built on by (most cited first, 5 of 5)

- [LLMs Will Always Hallucinate, and We Need to Live With This](https://mlsanity.com/paper/2409.05746/) (2024)
- [BLEND: Behavior-guided Neural Population Dynamics Modeling via Privileged Knowledge Distillation](https://mlsanity.com/paper/2410.13872/) (2024)
- [SKOLR: Structured Koopman Operator Linear RNN for Time-Series Forecasting](https://mlsanity.com/paper/2506.14113/) (2025)
- [Fast Classification Learning with Neural Networks and Conceptors for Speech Recognition and Car Driving Maneuvers](https://mlsanity.com/paper/2102.05588/) (2021)
- [Linear Recurrent Memory Suffices to Distil a World-Model Policy for Robot Air Hockey](https://mlsanity.com/paper/2609.39151/) (2026)

## Abstract (arXiv)

Recurrent neural networks are a powerful means to cope with time series. We show how autoregressive linear, i.e., linearly activated recurrent neural networks (LRNNs) can approximate any time-dependent function f(t). The approximation can effectively be learned by simply solving a linear equation system; no backpropagation or similar methods are needed. Furthermore, and this is the main contribution of this paper, the size of an LRNN can be reduced significantly in one step after inspecting the spectrum of the network transition matrix, i.e., its eigenvalues, by taking only the most relevant components. Therefore, in contrast to other approaches, we do not only learn network weights but also the network architecture. LRNNs have interesting properties: They end up in ellipse trajectories in the long run and allow the prediction of further values and compact representations of functions. We demonstrate this by several case studies, among them multiple superimposed oscillators (MSO), robotic soccer (RoboCup), and stock price prediction. LRNNs outperform the previous state-of-the-art for the MSO task with a minimal number of units.