# Approximation Theory of Convolutional Architectures for Time Series Modelling

arXiv: https://arxiv.org/abs/2107.09355 · HTML: https://mlsanity.com/paper/2107.09355/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 33 references that are in the map.
- First posted on arXiv 20 Jul 2021.

## Builds on (most cited first, 11 of 11)

- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [WaveGlow: A Flow-based Generative Network for Speech Synthesis](https://mlsanity.com/paper/1811.00002/) (2018)
- [Efficient Neural Audio Synthesis](https://mlsanity.com/paper/1802.08435/) (2018)
- [Parallel WaveNet: Fast High-Fidelity Speech Synthesis](https://mlsanity.com/paper/1711.10433/) (2017)
- [Universality of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1805.10769/) (2018)
- [Comparative Study of CNN and RNN for Natural Language Processing](https://mlsanity.com/paper/1702.01923/) (2017)
- [Approximation and Non-parametric Estimation of ResNet-type Convolutional Neural Networks](https://mlsanity.com/paper/1903.10047/) (2019)
- [On the Curse of Memory in Recurrent Neural Networks: Approximation and Optimization Analysis](https://mlsanity.com/paper/2009.07799/) (2020)
- [Fast Wavenet Generation Algorithm](https://mlsanity.com/paper/1611.09482/) (2016)

## Built on by (most cited first, 10 of 10)

- [Approximation Rate of the Transformer Architecture for Sequence Modeling](https://mlsanity.com/paper/2305.18475/) (2023)
- [Theoretical Analysis of Inductive Biases in Deep Convolutional Networks](https://mlsanity.com/paper/2305.08404/) (2023)
- [A Brief Survey on the Approximation Theory for Sequence Modelling](https://mlsanity.com/paper/2302.13752/) (2023)
- [Inverse Approximation Theory for Nonlinear Recurrent Neural Networks](https://mlsanity.com/paper/2305.19190/) (2023)
- [On the Universal Approximation Property of Deep Fully Convolutional Neural Networks](https://mlsanity.com/paper/2211.14047/) (2022)
- [Forward and Inverse Approximation Theory for Linear Temporal Convolutional Networks](https://mlsanity.com/paper/2305.18478/) (2023)
- [Curriculum Design Helps Spiking Neural Networks to Classify Time Series](https://mlsanity.com/paper/2401.10257/) (2023)
- [Numerical Investigation of Sequence Modeling Theory using Controllable Memory Functions](https://mlsanity.com/paper/2506.05678/) (2025)
- [Performance-Efficiency Tradeoffs in Transformers: An Approximation Theory Perspective](https://mlsanity.com/paper/2510.03784/) (2025)
- [Dualformer: Efficient Feature Extractor for Complex-valued Blind Communication Signal Analysis](https://mlsanity.com/paper/2606.31352/) (2026)

## Abstract (arXiv)

We study the approximation properties of convolutional architectures applied to time series modelling, which can be formulated mathematically as a functional approximation problem. In the recurrent setting, recent results reveal an intricate connection between approximation efficiency and memory structures in the data generation process. In this paper, we derive parallel results for convolutional architectures, with WaveNet being a prime example. Our results reveal that in this new setting, approximation efficiency is not only characterised by memory, but also additional fine structures in the target relationship. This leads to a novel definition of spectrum-based regularity that measures the complexity of temporal relationships under the convolutional approximation scheme. These analyses provide a foundation to understand the differences between architectural choices for time series modelling and can give theoretically grounded guidance for practical applications.