# KVNN: Learnable Multi-Kernel Volterra Neural Networks

arXiv: https://arxiv.org/abs/2604.15141 · HTML: https://mlsanity.com/paper/2604.15141/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 37 references that are in the map.
- First posted on arXiv 16 Apr 2026.

## Builds on (most cited first, 17 of 17)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising](https://mlsanity.com/paper/1710.04026/) (2017)
- [Bilinear CNNs for Fine-grained Visual Recognition](https://mlsanity.com/paper/1504.07889/) (2015)
- [Compact Bilinear Pooling](https://mlsanity.com/paper/1511.06062/) (2015)
- [Tensor Regression Networks](https://mlsanity.com/paper/1707.08308/) (2017)
- [Pi-nets: Deep Polynomial Neural Networks](https://mlsanity.com/paper/2003.03828/) (2020)
- [Scalable Kernel Methods via Doubly Stochastic Gradients](https://mlsanity.com/paper/1407.5599/) (2014)
- [Spatially Adaptive Self-Supervised Learning for Real-World Image Denoising](https://mlsanity.com/paper/2303.14934/) (2023)
- [Video-FocalNets: Spatio-Temporal Focal Modulation for Video Action Recognition](https://mlsanity.com/paper/2307.06947/) (2023)
- [VolterraNet: A higher order convolutional network with group equivariance for homogeneous manifolds](https://mlsanity.com/paper/2106.15301/) (2021)
- [Volterra Neural Networks (VNNs)](https://mlsanity.com/paper/1910.09616/) (2019)
- [DCANet: Dual Convolutional Neural Network with Attention for Image Blind Denoising](https://mlsanity.com/paper/2304.01498/) (2023)
- [Latent Code-Based Fusion: A Volterra Neural Network Approach](https://mlsanity.com/paper/2104.04829/) (2021)
- [DVFL-Net: A Lightweight Distilled Video Focal Modulation Network for Spatio-Temporal Action Recognition](https://mlsanity.com/paper/2507.12426/) (2025)

## Abstract (arXiv)

Higher-order learning is fundamentally rooted in exploiting compositional features. It clearly hinges on enriching the representation by more elaborate interactions of the data which, in turn, tends to increase the model complexity of conventional large-scale deep learning models. In this paper, a kernelized Volterra Neural Network (kVNN) is proposed. The key to the achieved efficiency lies in using a learnable multi-kernel representation, where different interaction orders are modeled by distinct polynomial-kernel components with compact, learnable centers, yielding an order-adaptive parameterization. Features are learned by the composition of layers, each of which consists of parallel branches of different polynomial orders, enabling kVNN filters to directly replace standard convolutional kernels within existing architectures. The theoretical results are substantiated by experiments on two representative tasks: video action recognition and image denoising. The results demonstrate favorable performance-efficiency trade-offs: kVNN consistently yields reduced model (parameters) and computational (GFLOPs) complexity with competitive and often improved performance. These results are maintained even when trained from scratch without large-scale pretraining. In summary, we substantiate that structured kernelized higher-order layers offer a practical path to balancing expressivity and computational cost in modern deep networks.