# FastWave: Accelerating Autoregressive Convolutional Neural Networks on FPGA

arXiv: https://arxiv.org/abs/2002.04971 · HTML: https://mlsanity.com/paper/2002.04971/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Feb 2020.

## Built on by (most cited first, 4 of 4)

- [Sparse Attention Acceleration with Synergistic In-Memory Pruning and On-Chip Recomputation](https://mlsanity.com/paper/2209.00606/) (2022)
- [Accelerating Attention through Gradient-Based Learned Runtime Pruning](https://mlsanity.com/paper/2204.03227/) (2022)
- [MAFIA: Machine Learning Acceleration on FPGAs for IoT Applications](https://mlsanity.com/paper/2107.03653/) (2021)
- [CASSOD-Net: Cascaded and Separable Structures of Dilated Convolution for Embedded Vision Systems and Applications](https://mlsanity.com/paper/2104.14126/) (2021)

## Abstract (arXiv)

Autoregressive convolutional neural networks (CNNs) have been widely exploited for sequence generation tasks such as audio synthesis, language modeling and neural machine translation. WaveNet is a deep autoregressive CNN composed of several stacked layers of dilated convolution that is used for sequence generation. While WaveNet produces state-of-the art audio generation results, the naive inference implementation is quite slow; it takes a few minutes to generate just one second of audio on a high-end GPU. In this work, we develop the first accelerator platform~textitFastWave for autoregressive convolutional neural networks, and address the associated design challenges. We design the Fast-Wavenet inference model in Vivado HLS and perform a wide range of optimizations including fixed-point implementation, array partitioning and pipelining. Our model uses a fully parameterized parallel architecture for fast matrix-vector multiplication that enables per-layer customized latency fine-tuning for further throughput improvement. Our experiments comparatively assess the trade-off between throughput and resource utilization for various optimizations. Our best WaveNet design on the Xilinx XCVU13P FPGA that uses only on-chip memory, achieves 66 faster generation speed compared to CPU implementation and 11 faster generation speed than GPU implementation.