# 3D Segmentation with Fully Trainable Gabor Kernels and Pearson's Correlation Coefficient

arXiv: https://arxiv.org/abs/2201.03644 · HTML: https://mlsanity.com/paper/2201.03644/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 13 references that are in the map.
- First posted on arXiv 10 Jan 2022.

## Builds on (most cited first, 9 of 9)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Tversky loss function for image segmentation using 3D fully convolutional deep networks](https://mlsanity.com/paper/1706.05721/) (2017)
- [Efficient Object Localization Using Convolutional Networks](https://mlsanity.com/paper/1411.4280/) (2014)
- [Gabor Convolutional Networks](https://mlsanity.com/paper/1705.01450/) (2017)
- [3D Segmentation with Exponential Logarithmic Loss for Highly Unbalanced Object Sizes](https://mlsanity.com/paper/1809.00076/) (2018)
- [Gabor Filter Assisted Energy Efficient Fast Learning Convolutional Neural Networks](https://mlsanity.com/paper/1705.04748/) (2017)

## Built on by (most cited first, 5 of 5)

- [FNOSeg3D: Resolution-Robust 3D Image Segmentation with Fourier Neural Operator](https://mlsanity.com/paper/2310.03872/) (2023)
- [Learning Relative Gene Expression Trends from Pathology Images in Spatial Transcriptomics](https://mlsanity.com/paper/2512.06612/) (2025)
- [HartleyMHA: Self-Attention in Frequency Domain for Resolution-Robust and Parameter-Efficient 3D Image Segmentation](https://mlsanity.com/paper/2310.04466/) (2023)
- [HNOSeg-XS: Extremely Small Hartley Neural Operator for Efficient and Resolution-Robust 3D Image Segmentation](https://mlsanity.com/paper/2507.08205/) (2025)
- [HyperBank: A Differentiable Bank of Classical Priors for Few-Shot Spheroid Microscopy Segmentation](https://mlsanity.com/paper/2607.10684/) (2026)

## Abstract (arXiv)

The convolutional layer and loss function are two fundamental components in deep learning. Because of the success of conventional deep learning kernels, the less versatile Gabor kernels become less popular despite the fact that they can provide abundant features at different frequencies, orientations, and scales with much fewer parameters. For existing loss functions for multi-class image segmentation, there is usually a tradeoff among accuracy, robustness to hyperparameters, and manual weight selections for combining different losses. Therefore, to gain the benefits of using Gabor kernels while keeping the advantage of automatic feature generation in deep learning, we propose a fully trainable Gabor-based convolutional layer where all Gabor parameters are trainable through backpropagation. Furthermore, we propose a loss function based on the Pearson's correlation coefficient, which is accurate, robust to learning rates, and does not require manual weight selections. Experiments on 43 3D brain magnetic resonance images with 19 anatomical structures show that, using the proposed loss function with a proper combination of conventional and Gabor-based kernels, we can train a network with only 1.6 million parameters to achieve an average Dice coefficient of 83%. This size is 44 times smaller than the original V-Net which has 71 million parameters. This paper demonstrates the potentials of using learnable parametric kernels in deep learning for 3D segmentation.