# Super-Resolution Based Patch-Free 3D Image Segmentation with High-Frequency Guidance

arXiv: https://arxiv.org/abs/2210.14645 · HTML: https://mlsanity.com/paper/2210.14645/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 45 references that are in the map.
- First posted on arXiv 26 Oct 2022.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images](https://mlsanity.com/paper/2201.01266/) (2022)
- [Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge](https://mlsanity.com/paper/1811.02629/) (2018)
- [Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis](https://mlsanity.com/paper/2111.14791/) (2021)
- [UNet 3+: A Full-Scale Connected UNet for Medical Image Segmentation](https://mlsanity.com/paper/2004.08790/) (2020)
- [3D MRI brain tumor segmentation using autoencoder regularization](https://mlsanity.com/paper/1810.11654/) (2018)
- [Uncertainty-aware multi-view co-training for semi-supervised medical image segmentation and domain adaptation](https://mlsanity.com/paper/2006.16806/) (2020)
- [No New-Net](https://mlsanity.com/paper/1809.10483/) (2018)
- [JCS: An Explainable COVID-19 Diagnosis System by Joint Classification and Segmentation](https://mlsanity.com/paper/2004.07054/) (2020)
- [Abdominal multi-organ segmentation with organ-attention networks and statistical fusion](https://mlsanity.com/paper/1804.08414/) (2018)
- [Mind the Pad -- CNNs can Develop Blind Spots](https://mlsanity.com/paper/2010.02178/) (2020)
- [Simultaneous Super-Resolution and Cross-Modality Synthesis of 3D Medical Images using Weakly-Supervised Joint Convolutional Sparse Coding](https://mlsanity.com/paper/1705.02596/) (2017)
- [Improving RetinaNet for CT Lesion Detection with Dense Masks from Weak RECIST Labels](https://mlsanity.com/paper/1906.02283/) (2019)
- [Bi-directional Dermoscopic Feature Learning and Multi-scale Consistent Decision Fusion for Skin Lesion Segmentation](https://mlsanity.com/paper/2002.08694/) (2020)
- [E2Net: An Edge Enhanced Network for Accurate Liver and Tumor Segmentation on CT Scans](https://mlsanity.com/paper/2007.09791/) (2020)
- [Bidirectional RNN-based Few Shot Learning for 3D Medical Image Segmentation](https://mlsanity.com/paper/2011.09608/) (2020)
- [Point-Unet: A Context-aware Point-based Neural Network for Volumetric Segmentation](https://mlsanity.com/paper/2203.08964/) (2022)
- [VolumeNet: A Lightweight Parallel Network for Super-Resolution of Medical Volumetric Data](https://mlsanity.com/paper/2010.08357/) (2020)
- [Attentive CT Lesion Detection Using Deep Pyramid Inference with Multi-Scale Booster](https://mlsanity.com/paper/1907.03958/) (2019)

## Built on by (most cited first, 1 of 1)

- [Scale-aware Super-resolution Network with Dual Affinity Learning for Lesion Segmentation from Medical Images](https://mlsanity.com/paper/2305.19063/) (2023)

## Abstract (arXiv)

High resolution (HR) 3D images are widely used nowadays, such as medical images like Magnetic Resonance Imaging (MRI) and Computed Tomography (CT). However, segmentation of these 3D images remains a challenge due to their high spatial resolution and dimensionality in contrast to currently limited GPU memory. Therefore, most existing 3D image segmentation methods use patch-based models, which have low inference efficiency and ignore global contextual information. To address these problems, we propose a super-resolution (SR) based patch-free 3D image segmentation framework that can realize HR segmentation from a global-wise low-resolution (LR) input. The framework contains two sub-tasks, of which semantic segmentation is the main task and super resolution is an auxiliary task aiding in rebuilding the high frequency information from the LR input. To furthermore balance the information loss with the LR input, we propose a High-Frequency Guidance Module (HGM), and design an efficient selective cropping algorithm to crop an HR patch from the original image as restoration guidance for it. In addition, we also propose a Task-Fusion Module (TFM) to exploit the inter connections between segmentation and SR task, realizing joint optimization of the two tasks. When predicting, only the main segmentation task is needed, while other modules can be removed for acceleration. The experimental results on two different datasets show that our framework has a four times higher inference speed compared to traditional patch-based methods, while its performance also surpasses other patch-based and patch-free models.