mlsanityData to 14 Oct 2026

Dilated Recurrent Neural Networks

arXiv 1710.02224 ↗First posted 5 Oct 2017cs.AI cs.LGcited by 83

Not on the map. Why?

Abstract (arXiv)

Learning with recurrent neural networks (RNNs) on long sequences is a notoriously difficult task. There are three major challenges: 1) complex dependencies, 2) vanishing and exploding gradients, and 3) efficient parallelization. In this paper, we introduce a simple yet effective RNN connection structure, the DilatedRNN, which simultaneously tackles all of these challenges. The proposed architecture is characterized by multi-resolution dilated recurrent skip connections and can be combined flexibly with diverse RNN cells. Moreover, the DilatedRNN reduces the number of parameters needed and enhances training efficiency significantly, while matching state-of-the-art performance (even with standard RNN cells) in tasks involving very long-term dependencies. To provide a theory-based quantification of the architecture's advantages, we introduce a memory capacity measure, the mean recurrent length, which is more suitable for RNNs with long skip connections than existing measures. We rigorously prove the advantages of the DilatedRNN over other recurrent neural architectures. The code for our method is publicly available at https://github.com/code-terminator/DilatedRNN

Key facts. Cited by 83 papers in the map (data through 14 Oct 2026). First posted on arXiv 5 Oct 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 18; 2022: 20; 2023: 10; 2024: 15; 2025: 16; 2026: 4.

83 papers.

Sort:

By area:

  1. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  2. 2022Simplified State Space Layers for Sequence Modelingcited by 788
  3. 2021Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layerscited by 608
  4. 2022N-HiTS: Neural Hierarchical Interpolation for Time Series Forecastingcited by 146
  5. 2022Liquid Structural State-Space Modelscited by 136
  6. 2022Deep Learning for Time Series Anomaly Detection: A Surveycited by 131
  7. 2021CKConv: Continuous Kernel Convolution For Sequential Datacited by 120
  8. 2023Hierarchically Gated Recurrent Neural Network for Sequence Modelingcited by 102
  9. 2023Generative Time Series Forecasting with Diffusion, Denoise, and Disentanglementcited by 90
  10. 2025Dynamic Chunking for End-to-End Hierarchical Sequence Modelingcited by 72
  11. 2021FlexConv: Continuous Kernel Convolutions with Differentiable Kernel Sizescited by 66
  12. 2021Neural basis expansion analysis with exogenous variables: Forecasting electricity prices with NBEATSxcited by 64
38 more
  1. 2021UnICORNN: A recurrent model for learning very long time dependenciescited by 56
  2. 2021Variational Neural Annealingcited by 47
  3. 2021ScoreGrad: Multivariate Probabilistic Time Series Forecasting with Continuous Energy-based Generative Modelscited by 43
  4. 2022Dissecting Transformer Length Extrapolation via the Lens of Receptive Field Analysiscited by 42
  5. 2021Long Expressive Memory for Sequence Modelingcited by 41
  6. 2026ReMem-VLA: Empowering Vision-Language-Action Model with Memory via Dual-Level Recurrent Queriescited by 29
  7. 2022Unsupervised Model Selection for Time-series Anomaly Detectioncited by 25
  8. 2024Large Language Models are Zero-Shot Next Location Predictorscited by 22
  9. 2024Graph-based Forecasting with Missing Data through Spatiotemporal Downsamplingcited by 17
  10. 2021Recurrent Neural Network from Adder's Perspective: Carry-lookahead RNNcited by 16
  11. 2022Towards a General Purpose CNN for Long Range Dependencies in NDcited by 16
  12. 2021Recurrence along Depth: Deep Convolutional Neural Networks with Recurrent Layer Aggregationcited by 13
  13. 2021ES-dRNN: A Hybrid Exponential Smoothing and Dilated Recurrent Neural Network Model for Short-Term Load Forecastingcited by 11
  14. 2023Unsupervised Deep Learning for IoT Time Seriescited by 11
  15. 2022Time-Series Anomaly Detection with Implicit Neural Representationcited by 10
  16. 2023A Systematic Review for Transformer-based Long-term Series Forecastingcited by 10
  17. 2021Scale-Aware Neural Architecture Search for Multivariate Time Series Forecastingcited by 9
  18. 2022Path Development Network with Finite-dimensional Lie Group Representationcited by 8
  19. 2023SutraNets: Sub-series Autoregressive Networks for Long-Sequence, Probabilistic Forecastingcited by 7
  20. 2022Supplementing Recurrent Neural Networks with Annealing to Solve Combinatorial Optimization Problemscited by 6
  21. 2023SMPConv: Self-moving Point Representations for Continuous Convolutioncited by 6
  22. 2024Message Passing Variational Autoregressive Network for Solving Intractable Ising Modelscited by 6
  23. 2022Random orthogonal additive filters: a solution to the vanishing/exploding gradient of deep neural networkscited by 5
  24. 2022Gated Recurrent Neural Networks with Weighted Time-Delay Feedbackcited by 5
  25. 2022Deep Temporal Contrastive Clusteringcited by 5
  26. 2023Text Analysis Using Deep Neural Networks in Digital Humanities and Information Sciencecited by 5
  27. 2024Positional Encoding Helps Recurrent Neural Networks Handle a Large Vocabularycited by 5
  28. 2024PGN: The RNN's New Successor is Effective for Long-Range Time Series Forecastingcited by 4
  29. 2025Emergence of the Primacy Effect in Structured State-Space Modelscited by 4
  30. 2021Hybrid Data Augmentation and Deep Attention-based Dilated Convolutional-Recurrent Neural Networks for Speech Emotion Recognitioncited by 3
  31. 2022Trajectory Test-Train Overlap in Next-Location Prediction Datasetscited by 3
  32. 2022Evaluating State of the Art, Forecasting Ensembles- and Meta-learning Strategies for Model Fusioncited by 3
  33. 2022Persistence Initialization: A novel adaptation of the Transformer architecture for Time Series Forecastingcited by 3
  34. 2024Time Series Foundation Models and Deep Learning Architectures for Earthquake Temporal and Spatial Nowcastingcited by 3
  35. 2024Deep Learning Foundation and Pattern Models: Challenges in Hydrological Time Seriescited by 3
  36. 2025Lattice Protein Folding with Variational Annealingcited by 3
  37. 2021Oscillatory Fourier Neural Network: A Compact and Efficient Architecture for Sequential Processingcited by 2
  38. 2022Comparison and Analysis of New Curriculum Criteria for End-to-End ASRcited by 2

50 most cited of 83 All 83 (JSON)