# Dilated Recurrent Neural Networks

arXiv: https://arxiv.org/abs/1710.02224 · HTML: https://mlsanity.com/paper/1710.02224/

## Key facts

- Cited by 83 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Oct 2017.

## Built on by (most cited first, 30 of 83)

- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Simplified State Space Layers for Sequence Modeling](https://mlsanity.com/paper/2208.04933/) (2022)
- [Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers](https://mlsanity.com/paper/2110.13985/) (2021)
- [N-HiTS: Neural Hierarchical Interpolation for Time Series Forecasting](https://mlsanity.com/paper/2201.12886/) (2022)
- [Liquid Structural State-Space Models](https://mlsanity.com/paper/2209.12951/) (2022)
- [Deep Learning for Time Series Anomaly Detection: A Survey](https://mlsanity.com/paper/2211.05244/) (2022)
- [CKConv: Continuous Kernel Convolution For Sequential Data](https://mlsanity.com/paper/2102.02611/) (2021)
- [Hierarchically Gated Recurrent Neural Network for Sequence Modeling](https://mlsanity.com/paper/2311.04823/) (2023)
- [Generative Time Series Forecasting with Diffusion, Denoise, and Disentanglement](https://mlsanity.com/paper/2301.03028/) (2023)
- [Dynamic Chunking for End-to-End Hierarchical Sequence Modeling](https://mlsanity.com/paper/2507.07955/) (2025)
- [FlexConv: Continuous Kernel Convolutions with Differentiable Kernel Sizes](https://mlsanity.com/paper/2110.08059/) (2021)
- [Neural basis expansion analysis with exogenous variables: Forecasting electricity prices with NBEATSx](https://mlsanity.com/paper/2104.05522/) (2021)
- [UnICORNN: A recurrent model for learning very long time dependencies](https://mlsanity.com/paper/2103.05487/) (2021)
- [Variational Neural Annealing](https://mlsanity.com/paper/2101.10154/) (2021)
- [ScoreGrad: Multivariate Probabilistic Time Series Forecasting with Continuous Energy-based Generative Models](https://mlsanity.com/paper/2106.10121/) (2021)
- [Dissecting Transformer Length Extrapolation via the Lens of Receptive Field Analysis](https://mlsanity.com/paper/2212.10356/) (2022)
- [Long Expressive Memory for Sequence Modeling](https://mlsanity.com/paper/2110.04744/) (2021)
- [ReMem-VLA: Empowering Vision-Language-Action Model with Memory via Dual-Level Recurrent Queries](https://mlsanity.com/paper/2603.12942/) (2026)
- [Unsupervised Model Selection for Time-series Anomaly Detection](https://mlsanity.com/paper/2210.01078/) (2022)
- [Large Language Models are Zero-Shot Next Location Predictors](https://mlsanity.com/paper/2405.20962/) (2024)
- [Graph-based Forecasting with Missing Data through Spatiotemporal Downsampling](https://mlsanity.com/paper/2402.10634/) (2024)
- [Recurrent Neural Network from Adder's Perspective: Carry-lookahead RNN](https://mlsanity.com/paper/2106.12901/) (2021)
- [Towards a General Purpose CNN for Long Range Dependencies in ND](https://mlsanity.com/paper/2206.03398/) (2022)
- [Recurrence along Depth: Deep Convolutional Neural Networks with Recurrent Layer Aggregation](https://mlsanity.com/paper/2110.11852/) (2021)
- [ES-dRNN: A Hybrid Exponential Smoothing and Dilated Recurrent Neural Network Model for Short-Term Load Forecasting](https://mlsanity.com/paper/2112.02663/) (2021)
- [Unsupervised Deep Learning for IoT Time Series](https://mlsanity.com/paper/2302.03284/) (2023)
- [Time-Series Anomaly Detection with Implicit Neural Representation](https://mlsanity.com/paper/2201.11950/) (2022)
- [A Systematic Review for Transformer-based Long-term Series Forecasting](https://mlsanity.com/paper/2310.20218/) (2023)
- [Scale-Aware Neural Architecture Search for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2112.07459/) (2021)
- [Path Development Network with Finite-dimensional Lie Group Representation](https://mlsanity.com/paper/2204.00740/) (2022)

## Abstract (arXiv)

Learning with recurrent neural networks (RNNs) on long sequences is a notoriously difficult task. There are three major challenges: 1) complex dependencies, 2) vanishing and exploding gradients, and 3) efficient parallelization. In this paper, we introduce a simple yet effective RNN connection structure, the DilatedRNN, which simultaneously tackles all of these challenges. The proposed architecture is characterized by multi-resolution dilated recurrent skip connections and can be combined flexibly with diverse RNN cells. Moreover, the DilatedRNN reduces the number of parameters needed and enhances training efficiency significantly, while matching state-of-the-art performance (even with standard RNN cells) in tasks involving very long-term dependencies. To provide a theory-based quantification of the architecture's advantages, we introduce a memory capacity measure, the mean recurrent length, which is more suitable for RNNs with long skip connections than existing measures. We rigorously prove the advantages of the DilatedRNN over other recurrent neural architectures. The code for our method is publicly available at https://github.com/code-terminator/DilatedRNN