# Scaling Up 3D Kernels with Bayesian Frequency Re-parameterization for Medical Image Segmentation arXiv: https://arxiv.org/abs/2303.05785 · HTML: https://mlsanity.com/paper/2303.05785/ ## Key facts - Cited by 12 papers in the map (data through 14 Oct 2026). - Builds on 19 of its 25 references that are in the map. - First posted on arXiv 10 Mar 2023. ## Builds on (most cited first, 19 of 19) - [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020) - [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021) - [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022) - [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021) - [Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images](https://mlsanity.com/paper/2201.01266/) (2022) - [The Medical Segmentation Decathlon](https://mlsanity.com/paper/2106.05735/) (2021) - [RepVGG: Making VGG-style ConvNets Great Again](https://mlsanity.com/paper/2101.03697/) (2021) - [The Liver Tumor Segmentation Benchmark (LiTS)](https://mlsanity.com/paper/1901.04056/) (2019) - [Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis](https://mlsanity.com/paper/2111.14791/) (2021) - [AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation](https://mlsanity.com/paper/2206.08023/) (2022) - [TransBTS: Multimodal Brain Tumor Segmentation Using Transformer](https://mlsanity.com/paper/2103.04430/) (2021) - [AbdomenCT-1K: Is Abdominal Organ Segmentation A Solved Problem?](https://mlsanity.com/paper/2010.14808/) (2020) - [DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation](https://mlsanity.com/paper/1506.06448/) (2015) - [Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs](https://mlsanity.com/paper/2203.06717/) (2022) - [More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity](https://mlsanity.com/paper/2207.03620/) (2022) - [3D UX-Net: A Large Kernel Volumetric ConvNet Modernizing Hierarchical Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2209.15076/) (2022) - [Re-parameterizing Your Optimizers rather than Architectures](https://mlsanity.com/paper/2205.15242/) (2022) - [Large-Kernel Attention for 3D Medical Image Segmentation](https://mlsanity.com/paper/2207.11225/) (2022) - [Online Convolutional Re-parameterization](https://mlsanity.com/paper/2204.00826/) (2022) ## Built on by (most cited first, 12 of 12) - [BiomedParse: a biomedical foundation model for image parsing of everything everywhere all at once](https://mlsanity.com/paper/2405.12971/) (2024) - [Foundation Models for Biomedical Image Segmentation: A Survey](https://mlsanity.com/paper/2401.07654/) (2024) - [Rethinking Abdominal Organ Segmentation (RAOS) in the clinical scenario: A robustness evaluation benchmark with challenging cases](https://mlsanity.com/paper/2406.13674/) (2024) - [3D-MIR: A Benchmark and Empirical Study on 3D Medical Image Retrieval in Radiology](https://mlsanity.com/paper/2311.13752/) (2023) - [DeformUX-Net: Exploring a 3D Foundation Backbone for Medical Image Segmentation with Depthwise Deformable Convolution](https://mlsanity.com/paper/2310.00199/) (2023) - [Enhancing Single-Slice Segmentation with 3D-to-2D Unpaired Scan Distillation](https://mlsanity.com/paper/2406.12254/) (2024) - [Enhancing Hierarchical Transformers for Whole Brain Segmentation with Intracranial Measurements Integration](https://mlsanity.com/paper/2309.04071/) (2023) - [Efficient 3D affinely equivariant CNNs with adaptive fusion of augmented spherical Fourier-Bessel bases](https://mlsanity.com/paper/2402.16825/) (2024) - [Swin SMT: Global Sequential Modeling in 3D Medical Image Segmentation](https://mlsanity.com/paper/2407.07514/) (2024) - [Spatially-Adaptive Gradient Re-parameterization for 3D Large Kernel Optimization](https://mlsanity.com/paper/2505.19603/) (2025) - [Partial Decoder Attention Network with Contour-weighted Loss Function for Data-Imbalance Medical Image Segmentation](https://mlsanity.com/paper/2601.14338/) (2026) - [Knowledge Transfer Scaling Laws for 3D Medical Imaging](https://mlsanity.com/paper/2605.06859/) (2026) ## Abstract (arXiv) With the inspiration of vision transformers, the concept of depth-wise convolution revisits to provide a large Effective Receptive Field (ERF) using Large Kernel (LK) sizes for medical image segmentation. However, the segmentation performance might be saturated and even degraded as the kernel sizes scaled up (e.g., 21× 21× 21) in a Convolutional Neural Network (CNN). We hypothesize that convolution with LK sizes is limited to maintain an optimal convergence for locality learning. While Structural Re-parameterization (SR) enhances the local convergence with small kernels in parallel, optimal small kernel branches may hinder the computational efficiency for training. In this work, we propose RepUX-Net, a pure CNN architecture with a simple large kernel block design, which competes favorably with current network state-of-the-art (SOTA) (e.g., 3D UX-Net, SwinUNETR) using 6 challenging public datasets. We derive an equivalency between kernel re-parameterization and the branch-wise variation in kernel convergence. Inspired by the spatial frequency in the human visual system, we extend to vary the kernel convergence into element-wise setting and model the spatial frequency as a Bayesian prior to re-parameterize convolutional weights during training. Specifically, a reciprocal function is leveraged to estimate a frequency-weighted value, which rescales the corresponding kernel element for stochastic gradient descent. From the experimental results, RepUX-Net consistently outperforms 3D SOTA benchmarks with internal validation (FLARE: 0.929 to 0.944), external validation (MSD: 0.901 to 0.932, KiTS: 0.815 to 0.847, LiTS: 0.933 to 0.949, TCIA: 0.736 to 0.779) and transfer learning (AMOS: 0.880 to 0.911) scenarios in Dice Score.