# 3D Anisotropic Hybrid Network: Transferring Convolutional Features from 2D Images to 3D Anisotropic Volumes

arXiv: https://arxiv.org/abs/1711.08580 · HTML: https://mlsanity.com/paper/1711.08580/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Nov 2017.

## Built on by (most cited first, 30 of 38)

- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [Modality-aware Mutual Learning for Multi-modal Medical Image Segmentation](https://mlsanity.com/paper/2107.09842/) (2021)
- [Federated Learning for Medical Applications: A Taxonomy, Current Trends, Challenges, and Future Research Directions](https://mlsanity.com/paper/2208.03392/) (2022)
- [More than Encoder: Introducing Transformer Decoder to Upsample](https://mlsanity.com/paper/2106.10637/) (2021)
- [The state-of-the-art 3D anisotropic intracranial hemorrhage segmentation on non-contrast head CT: The INSTANCE challenge](https://mlsanity.com/paper/2301.03281/) (2023)
- [Medical Image Segmentation with Limited Supervision: A Review of Deep Network Models](https://mlsanity.com/paper/2103.00429/) (2021)
- [MagicNet: Semi-Supervised Multi-Organ Segmentation via Magic-Cube Partition and Recovery](https://mlsanity.com/paper/2212.14310/) (2022)
- [Devil is in the Queries: Advancing Mask Transformers for Real-world Medical Image Segmentation and Out-of-Distribution Localization](https://mlsanity.com/paper/2304.00212/) (2023)
- [Semi-Supervised Segmentation of Radiation-Induced Pulmonary Fibrosis from Lung CT Scans with Multi-Scale Guided Dense Attention](https://mlsanity.com/paper/2109.14172/) (2021)
- [HyperSegNAS: Bridging One-Shot Neural Architecture Search with 3D Medical Image Segmentation using HyperNet](https://mlsanity.com/paper/2112.10652/) (2021)
- [T-AutoML: Automated Machine Learning for Lesion Segmentation using Transformers in 3D Medical Imaging](https://mlsanity.com/paper/2111.07535/) (2021)
- [A Novel Hybrid Convolutional Neural Network for Accurate Organ Segmentation in 3D Head and Neck CT Images](https://mlsanity.com/paper/2109.12634/) (2021)
- [Data efficient deep learning for medical image analysis: A survey](https://mlsanity.com/paper/2310.06557/) (2023)
- [Multi-Slice Low-Rank Tensor Decomposition Based Multi-Atlas Segmentation: Application to Automatic Pathological Liver CT Segmentation](https://mlsanity.com/paper/2102.12056/) (2021)
- [Automated liver tissues delineation techniques: A systematic survey on machine learning current trends and future orientations](https://mlsanity.com/paper/2103.06384/) (2021)
- [3DConvCaps: 3DUnet with Convolutional Capsule Encoder for Medical Image Segmentation](https://mlsanity.com/paper/2205.09299/) (2022)
- [Boosting Convolution with Efficient MLP-Permutation for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/2303.13111/) (2023)
- [BGDB: Bernoulli-Gaussian Decision Block with Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2409.13116/) (2024)
- [Conquering Data Variations in Resolution: A Slice-Aware Multi-Branch Decoder Network](https://mlsanity.com/paper/2203.03640/) (2022)
- [3D Shuffle-Mixer: An Efficient Context-Aware Vision Learner of Transformer-MLP Paradigm for Dense Prediction in Medical Volume](https://mlsanity.com/paper/2204.06779/) (2022)
- [Decoupled Pyramid Correlation Network for Liver Tumor Segmentation from CT images](https://mlsanity.com/paper/2205.13199/) (2022)
- [Multi-organ segmentation: a progressive exploration of learning paradigms under scarce annotation](https://mlsanity.com/paper/2302.03296/) (2023)
- [3D U-NetR: Low Dose Computed Tomography Reconstruction via Deep Learning and 3 Dimensional Convolutions](https://mlsanity.com/paper/2105.14130/) (2021)
- [A survey on deep learning approaches for breast cancer diagnosis](https://mlsanity.com/paper/2109.08853/) (2021)
- [Slice Imputation: Intermediate Slice Interpolation for Anisotropic 3D Medical Image Segmentation](https://mlsanity.com/paper/2203.10773/) (2022)
- [Moving from 2D to 3D: volumetric medical image classification for rectal cancer staging](https://mlsanity.com/paper/2209.05771/) (2022)
- [Seeing Beyond Cancer: Multi-Institutional Validation of Object Localization and 3D Semantic Segmentation using Deep Learning for Breast MRI](https://mlsanity.com/paper/2311.16213/) (2023)
- [Q2A: Querying Implicit Fully Continuous Feature Pyramid to Align Features for Medical Image Segmentation](https://mlsanity.com/paper/2404.09472/) (2024)
- [MorphoSeg: An Uncertainty-Aware Deep Learning Method for Biomedical Segmentation of Complex Cellular Morphologies](https://mlsanity.com/paper/2409.17110/) (2024)
- [COVID-19 Detection in Computed Tomography Images with 2D and 3D Approaches](https://mlsanity.com/paper/2105.08506/) (2021)

## Abstract (arXiv)

While deep convolutional neural networks (CNN) have been successfully applied for 2D image analysis, it is still challenging to apply them to 3D anisotropic volumes, especially when the within-slice resolution is much higher than the between-slice resolution and when the amount of 3D volumes is relatively small. On one hand, direct learning of CNN with 3D convolution kernels suffers from the lack of data and likely ends up with poor generalization; insufficient GPU memory limits the model size or representational power. On the other hand, applying 2D CNN with generalizable features to 2D slices ignores between-slice information. Coupling 2D network with LSTM to further handle the between-slice information is not optimal due to the difficulty in LSTM learning. To overcome the above challenges, we propose a 3D Anisotropic Hybrid Network (AH-Net) that transfers convolutional features learned from 2D images to 3D anisotropic volumes. Such a transfer inherits the desired strong generalization capability for within-slice information while naturally exploiting between-slice information for more effective modelling. The focal loss is further utilized for more effective end-to-end learning. We experiment with the proposed 3D AH-Net on two different medical image analysis tasks, namely lesion detection from a Digital Breast Tomosynthesis volume, and liver and liver tumor segmentation from a Computed Tomography volume and obtain the state-of-the-art results.