mlsanityData to 7 Oct 2026

An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling

arXiv 1803.01271 ↗First posted 4 Mar 2018cs.LG cs.AI cs.CLcited by 1,447

Not on the map. Why?

Abstract (arXiv)

For most deep learning practitioners, sequence modeling is synonymous with recurrent networks. Yet recent results indicate that convolutional architectures can outperform recurrent networks on tasks such as audio synthesis and machine translation. Given a new sequence modeling task or dataset, which architecture should one use? We conduct a systematic evaluation of generic convolutional and recurrent architectures for sequence modeling. The models are evaluated across a broad range of standard tasks that are commonly used to benchmark recurrent networks. Our results indicate that a simple convolutional architecture outperforms canonical recurrent networks such as LSTMs across a diverse range of tasks and datasets, while demonstrating longer effective memory. We conclude that the common association between sequence modeling and recurrent networks should be reconsidered, and convolutional networks should be regarded as a natural starting point for sequence modeling tasks. To assist related work, we have made code available at http://github.com/locuslab/TCN .

Key facts. Cited by 1,447 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 1; 2021: 174; 2022: 189; 2023: 250; 2024: 251; 2025: 301; 2026: 280.

1,447 papers.

Sort:

By area:

  1. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  2. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  3. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
  4. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,400
  5. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 906
  6. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 814
  7. 2022Simplified State Space Layers for Sequence Modelingcited by 776
  8. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 680
  9. 2021Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layerscited by 607
  10. 2021TS2Vec: Towards Universal Representation of Time Seriescited by 463
  11. 2023Resurrecting Recurrent Neural Networks for Long Sequencescited by 397
  12. 2021SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interactioncited by 335
38 more
  1. 2022Behavior Transformers: Cloning k modes with one stonecited by 299
  2. 2021Spectral Temporal Graph Neural Network for Multivariate Time-series Forecastingcited by 298
  3. 2024TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methodscited by 242
  4. 2021Unsupervised Representation Learning for Time Series with Temporal Neighborhood Codingcited by 223
  5. 2022Visual Attention Networkcited by 210
  6. 2024Timer: Generative Pre-trained Transformers Are Large Time Series Modelscited by 198
  7. 2024Deep Time Series Models: A Comprehensive Survey and Benchmarkcited by 191
  8. 2022BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesiscited by 177
  9. 2023TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecastingcited by 170
  10. 2023Frequency-domain MLPs are More Effective Learners in Time Series Forecastingcited by 170
  11. 2025Sundial: A Family of Highly Capable Time Series Foundation Modelscited by 168
  12. 2022Less Is More: Fast Multivariate Time Series Forecasting with Light Sampling-oriented MLP Structurescited by 158
  13. 2023TimeGPT-1cited by 149
  14. 2022CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecastingcited by 143
  15. 2021Graph Neural Controlled Differential Equations for Traffic Forecastingcited by 133
  16. 2023Real-World Humanoid Locomotion with Reinforcement Learningcited by 132
  17. 2022Deep Learning for Time Series Anomaly Detection: A Surveycited by 130
  18. 2023Taming Diffusion Models for Audio-Driven Co-Speech Gesture Generationcited by 128
  19. 2023FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspectivecited by 126
  20. 2021CKConv: Continuous Kernel Convolution For Sequential Datacited by 117
  21. 2023Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learningcited by 114
  22. 2023Koopa: Learning Non-stationary Time Series Dynamics with Koopman Predictorscited by 110
  23. 2021Audio2Gestures: Generating Diverse Gestures from Speech Audio with Conditional Variational Autoencoderscited by 109
  24. 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 109
  25. 2022Learning Hierarchical Cross-Modal Association for Co-Speech Gesture Generationcited by 106
  26. 2024CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuningcited by 105
  27. 2024SparseTSF: Modeling Long-term Time Series Forecasting with 1k Parameterscited by 103
  28. 2024CycleNet: Enhancing Time Series Forecasting through Modeling Periodic Patternscited by 102
  29. 2023EMAGE: Towards Unified Holistic Co-Speech Gesture Generation via Expressive Masked Audio Gesture Modelingcited by 101
  30. 2023The Capacity and Robustness Trade-off: Revisiting the Channel Independent Strategy for Multivariate Time Series Forecastingcited by 100
  31. 2021SGCN:Sparse Graph Convolution Network for Pedestrian Trajectory Predictioncited by 97
  32. 2023Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospectscited by 97
  33. 2021Self-Supervised Transformer for Sparse and Irregularly Sampled Multivariate Clinical Time-Seriescited by 96
  34. 2022Unsupervised Time-Series Representation Learning with Iterative Bilinear Temporal-Spectral Fusioncited by 96
  35. 2021TPCN: Temporal Point Cloud Networks for Motion Forecastingcited by 94
  36. 2021Filling the G_ap_s: Multivariate Time Series Imputation by Graph Neural Networkscited by 93
  37. 2021Darts: User-Friendly Modern Machine Learning for Time Seriescited by 90
  38. 2023Generative Time Series Forecasting with Diffusion, Denoise, and Disentanglementcited by 89

50 most cited of 1,447 All 1,447 (JSON)