# WavPool: A New Block for Deep Neural Networks

arXiv: https://arxiv.org/abs/2306.08734 · HTML: https://mlsanity.com/paper/2306.08734/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 32 references that are in the map.
- First posted on arXiv 14 Jun 2023.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution](https://mlsanity.com/paper/1704.03915/) (2017)
- [Multi-level Wavelet-CNN for Image Restoration](https://mlsanity.com/paper/1805.07071/) (2018)
- [Convolutional Neural Networks at Constrained Time Cost](https://mlsanity.com/paper/1412.1710/) (2014)
- [Wave-ViT: Unifying Wavelet and Transformers for Visual Representation Learning](https://mlsanity.com/paper/2207.04978/) (2022)
- [Wavelet Convolutional Neural Networks](https://mlsanity.com/paper/1805.08620/) (2018)
- [Fully Learnable Deep Wavelet Transform for Unsupervised Monitoring of High-Frequency Time Series](https://mlsanity.com/paper/2105.00899/) (2021)
- [Quantum Neural Networks: Concepts, Applications, and Challenges](https://mlsanity.com/paper/2108.01468/) (2021)
- [Learning a Wavelet-like Auto-Encoder to Accelerate Deep Neural Networks](https://mlsanity.com/paper/1712.07493/) (2017)

## Abstract (arXiv)

Modern deep neural networks comprise many operational layers, such as dense or convolutional layers, which are often collected into blocks. In this work, we introduce a new, wavelet-transform-based network architecture that we call the multi-resolution perceptron: by adding a pooling layer, we create a new network block, the WavPool. The first step of the multi-resolution perceptron is transforming the data into its multi-resolution decomposition form by convolving the input data with filters of fixed coefficients but increasing size. Following image processing techniques, we are able to make scale and spatial information simultaneously accessible to the network without increasing the size of the data vector. WavPool outperforms a similar multilayer perceptron while using fewer parameters, and outperforms a comparable convolutional neural network by ~ 10% on relative accuracy on CIFAR-10.