# PURe: A Plug-and-Play Product-Unit Residual Module for Vision Networks

arXiv: https://arxiv.org/abs/2505.04397 · HTML: https://mlsanity.com/paper/2505.04397/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 31 references that are in the map.
- First posted on arXiv 7 May 2025.

## Builds on (most cited first, 11 of 11)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation](https://mlsanity.com/paper/2206.08023/) (2022)
- [MixConv: Mixed Depthwise Convolutional Kernels](https://mlsanity.com/paper/1907.09595/) (2019)
- [Deep Polynomial Neural Networks](https://mlsanity.com/paper/2006.13026/) (2020)
- [Predicting nuclear masses with product-unit networks](https://mlsanity.com/paper/2305.04675/) (2023)
- [Convolution goes higher-order: a biologically inspired mechanism empowers image classification](https://mlsanity.com/paper/2412.06740/) (2024)
- [Leveraging Product as an Activation Function in Deep Networks](https://mlsanity.com/paper/1810.08578/) (2018)

## Built on by (most cited first, 2 of 2)

- [Modeling Nonlinear Feature Interactions with Product-Unit Residual Networks](https://mlsanity.com/paper/2606.06861/) (2026)
- [Product units in gated recurrent units improve nuclear-mass prediction](https://mlsanity.com/paper/2606.06866/) (2026)

## Abstract (arXiv)

Modern vision networks are dominated by additive local transformations, whereas explicit multiplicative local interactions remain underexplored. Product units offer a direct approach to modeling such interactions, but their use in deep architectures has been limited by optimization instability. In this work, we propose PURe, a product-unit Residual Module for deep vision networks. PURe is built around a 2D product unit with a real-valued log-domain formulation that makes multiplicative local aggregation practical within deep residual hierarchies. The resulting module serves as a drop-in replacement for native residual units. We instantiate PURe in residual CNNs for image classification and in 2D residual encoder--decoder networks for slice-based segmentation on volumetric CT data. Across Galaxy10 DECaLS, ImageNet, and CIFAR-10, PURe consistently improves residual CNNs and yields a more favorable accuracy--parameter trade-off, allowing moderately deep models to match or surpass substantially deeper ResNet baselines with much smaller parameter budgets. On the AMOS benchmark, PURe also improves slice-based CT segmentation under 3D case-level evaluation. These results show that explicit multiplicative local interaction is a practical and effective design primitive for deep residual vision networks.