mlsanityData to 7 Oct 2026

Approximation Theory of Convolutional Architectures for Time Series Modelling

arXiv 2107.09355 ↗First posted 20 Jul 2021cs.LG stat.MLcited by 10

Not on the map. Why?

Abstract (arXiv)

We study the approximation properties of convolutional architectures applied to time series modelling, which can be formulated mathematically as a functional approximation problem. In the recurrent setting, recent results reveal an intricate connection between approximation efficiency and memory structures in the data generation process. In this paper, we derive parallel results for convolutional architectures, with WaveNet being a prime example. Our results reveal that in this new setting, approximation efficiency is not only characterised by memory, but also additional fine structures in the target relationship. This leads to a novel definition of spectrum-based regularity that measures the complexity of temporal relationships under the convolutional approximation scheme. These analyses provide a foundation to understand the differences between architectural choices for time series modelling and can give theoretically grounded guidance for practical applications.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 11 of its 33 references that are in the map. First posted on arXiv 20 Jul 2021.

Builds on

11 of 33 references.

References · 11

  1. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  2. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,447
  3. 2015Multi-Scale Context Aggregation by Dilated Convolutionscited by 1,128
  4. 2018WaveGlow: A Flow-based Generative Network for Speech Synthesiscited by 308
  5. 2018Efficient Neural Audio Synthesiscited by 218
  6. 2017Parallel WaveNet: Fast High-Fidelity Speech Synthesiscited by 214
  7. 2018Universality of Deep Convolutional Neural Networkscited by 176
  8. 2017Comparative Study of CNN and RNN for Natural Language Processingcited by 81
  9. 2019Approximation and Non-parametric Estimation of ResNet-type Convolutional Neural Networkscited by 40
  10. 2020On the Curse of Memory in Recurrent Neural Networks: Approximation and Optimization Analysiscited by 26
  11. 2016Fast Wavenet Generation Algorithmcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 6; 2024: 0; 2025: 2; 2026: 1.

10 papers.

  1. 2023Approximation Rate of the Transformer Architecture for Sequence Modelingcited by 28
  2. 2023Theoretical Analysis of Inductive Biases in Deep Convolutional Networkscited by 21
  3. 2023A Brief Survey on the Approximation Theory for Sequence Modellingcited by 15
  4. 2023Inverse Approximation Theory for Nonlinear Recurrent Neural Networkscited by 10
  5. 2022On the Universal Approximation Property of Deep Fully Convolutional Neural Networkscited by 6
  6. 2023Forward and Inverse Approximation Theory for Linear Temporal Convolutional Networkscited by 1
  7. 2025Numerical Investigation of Sequence Modeling Theory using Controllable Memory Functionscited by 1
  8. 2023Curriculum Design Helps Spiking Neural Networks to Classify Time Seriescited by 0
  9. 2025Performance-Efficiency Tradeoffs in Transformers: An Approximation Theory Perspectivecited by 0
  10. 2026Dualformer: Efficient Feature Extractor for Complex-valued Blind Communication Signal Analysiscited by 0