# Convolution goes higher-order: a biologically inspired mechanism empowers image classification

arXiv: https://arxiv.org/abs/2412.06740 · HTML: https://mlsanity.com/paper/2412.06740/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 47 references that are in the map.
- First posted on arXiv 9 Dec 2024.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Image Quality Assessment: Unifying Structure and Texture Similarity](https://mlsanity.com/paper/2004.07728/) (2020)
- [Bilinear CNNs for Fine-grained Visual Recognition](https://mlsanity.com/paper/1504.07889/) (2015)
- [Compact Bilinear Pooling](https://mlsanity.com/paper/1511.06062/) (2015)
- [Non-linear Convolution Filters for CNN-based Learning](https://mlsanity.com/paper/1708.07038/) (2017)
- [Second-order Convolutional Neural Networks](https://mlsanity.com/paper/1703.06817/) (2017)
- [Toward Understanding Convolutional Neural Networks from Volterra Convolution Perspective](https://mlsanity.com/paper/2110.09902/) (2021)

## Built on by (most cited first, 4 of 4)

- [PURe: A Plug-and-Play Product-Unit Residual Module for Vision Networks](https://mlsanity.com/paper/2505.04397/) (2025)
- [Higher-Order Convolution Improves Neural Predictivity in the Retina](https://mlsanity.com/paper/2505.07620/) (2025)
- [Revisiting Neural Processes via Fourier Transform and Volterra Series](https://mlsanity.com/paper/2606.01172/) (2026)
- [Higher-Order Fourier Neural Operator: Explicit Mode Mixer for Nonlinear PDEs](https://mlsanity.com/paper/2606.28122/) (2026)

## Abstract (arXiv)

We propose a novel approach to image classification inspired by complex nonlinear biological visual processing, whereby classical convolutional neural networks (CNNs) are equipped with learnable higher-order convolutions. Our model incorporates a Volterra-like expansion of the convolution operator, capturing multiplicative interactions akin to those observed in early and advanced stages of biological visual processing. We evaluated this approach on synthetic datasets by measuring sensitivity to testing higher-order correlations and performance in standard benchmarks (MNIST, FashionMNIST, CIFAR10, CIFAR100 and Imagenette). Our architecture outperforms traditional CNN baselines, and achieves optimal performance with expansions up to 3rd/4th order, aligning remarkably well with the distribution of pixel intensities in natural images. Through systematic perturbation analysis, we validate this alignment by isolating the contributions of specific image statistics to model performance, demonstrating how different orders of convolution process distinct aspects of visual information. Furthermore, Representational Similarity Analysis reveals distinct geometries across network layers, indicating qualitatively different modes of visual information processing. Our work bridges neuroscience and deep learning, offering a path towards more effective, biologically inspired computer vision models. It provides insights into visual information processing and lays the groundwork for neural networks that better capture complex visual patterns, particularly in resource-constrained scenarios.