# The Statistical Recurrent Unit

arXiv: https://arxiv.org/abs/1703.00381 · HTML: https://mlsanity.com/paper/1703.00381/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Mar 2017.

## Built on by (most cited first, 10 of 10)

- [Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems](https://mlsanity.com/paper/2312.15234/) (2023)
- [Causal Discovery from Temporal Data: An Overview and New Perspectives](https://mlsanity.com/paper/2303.10112/) (2023)
- [CUTS: Neural Causal Discovery from Irregular Time-Series Data](https://mlsanity.com/paper/2302.07458/) (2023)
- [Causality-Aware Spatiotemporal Graph Neural Networks for Spatiotemporal Time Series Imputation](https://mlsanity.com/paper/2403.11960/) (2024)
- [Deep Recurrent Modelling of Granger Causality with Latent Confounding](https://mlsanity.com/paper/2202.11286/) (2022)
- [PID-Inspired Inductive Biases for Deep Reinforcement Learning in Partially Observable Control Tasks](https://mlsanity.com/paper/2307.05891/) (2023)
- [Combining Slow and Fast: Complementary Filtering for Dynamics Learning](https://mlsanity.com/paper/2302.13754/) (2023)
- [Learning Hybrid Dynamics Models With Simulator-Informed Latent States](https://mlsanity.com/paper/2309.02873/) (2023)
- [Bayesian Vector AutoRegression with Factorised Granger-Causal Graphs](https://mlsanity.com/paper/2402.03614/) (2024)
- [Coherent Probabilistic Aggregate Queries on Long-horizon Forecasts](https://mlsanity.com/paper/2111.03394/) (2021)

## Abstract (arXiv)

Sophisticated gated recurrent neural network architectures like LSTMs and GRUs have been shown to be highly effective in a myriad of applications. We develop an un-gated unit, the statistical recurrent unit (SRU), that is able to learn long term dependencies in data by only keeping moving averages of statistics. The SRU's architecture is simple, un-gated, and contains a comparable number of parameters to LSTMs; yet, SRUs perform favorably to more sophisticated LSTM and GRU alternatives, often outperforming one or both in various tasks. We show the efficacy of SRUs as compared to LSTMs and GRUs in an unbiased manner by optimizing respective architectures' hyperparameters in a Bayesian optimization scheme for both synthetic and real-world tasks.