# NiftyNet: a deep-learning platform for medical imaging

arXiv: https://arxiv.org/abs/1709.03485 · HTML: https://mlsanity.com/paper/1709.03485/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Sep 2017.

## Built on by (most cited first, 30 of 45)

- [MONAI: An open-source framework for deep learning in healthcare](https://mlsanity.com/paper/2211.02701/) (2022)
- [Transformers in Medical Imaging: A Survey](https://mlsanity.com/paper/2201.09873/) (2022)
- [Recent advances and clinical applications of deep learning in medical image analysis](https://mlsanity.com/paper/2105.13381/) (2021)
- [Transferable Visual Words: Exploiting the Semantics of Anatomical Patterns for Self-supervised Learning](https://mlsanity.com/paper/2102.10680/) (2021)
- [LVM-Med: Learning Large-Scale Self-Supervised Vision Models for Medical Imaging via Second-order Graph Matching](https://mlsanity.com/paper/2306.11925/) (2023)
- [GaNDLF: A Generally Nuanced Deep Learning Framework for Scalable End-to-End Clinical Workflows in Medical Imaging](https://mlsanity.com/paper/2103.01006/) (2021)
- [ExAID: A Multimodal Explanation Framework for Computer-Aided Diagnosis of Skin Lesions](https://mlsanity.com/paper/2201.01249/) (2022)
- [Medical Transformer: Universal Brain Encoder for 3D MRI Analysis](https://mlsanity.com/paper/2104.13633/) (2021)
- [Joint Self-Supervised Image-Volume Representation Learning with Intra-Inter Contrastive Clustering](https://mlsanity.com/paper/2212.01893/) (2022)
- [PyMIC: A deep learning toolkit for annotation-efficient medical image segmentation](https://mlsanity.com/paper/2208.09350/) (2022)
- [medigan: a Python library of pretrained generative models for medical image synthesis](https://mlsanity.com/paper/2209.14472/) (2022)
- [SegmentAnyBone: A Universal Model that Segments Any Bone at Any Location on MRI](https://mlsanity.com/paper/2401.12974/) (2024)
- [Primitive Geometry Segment Pre-training for 3D Medical Image Segmentation](https://mlsanity.com/paper/2401.03665/) (2024)
- [Learning Multi-Modal Volumetric Prostate Registration with Weak Inter-Subject Spatial Correspondence](https://mlsanity.com/paper/2102.04938/) (2021)
- [Interactive Segmentation via Deep Learning and B-Spline Explicit Active Surfaces](https://mlsanity.com/paper/2110.12939/) (2021)
- [Video Pretraining Advances 3D Deep Learning on Chest CT Tasks](https://mlsanity.com/paper/2304.00546/) (2023)
- [Automated liver tissues delineation techniques: A systematic survey on machine learning current trends and future orientations](https://mlsanity.com/paper/2103.06384/) (2021)
- [Estimating MRI Image Quality via Image Reconstruction Uncertainty](https://mlsanity.com/paper/2106.10992/) (2021)
- [A Decoupled Uncertainty Model for MRI Segmentation Quality Estimation](https://mlsanity.com/paper/2109.02413/) (2021)
- [Advancing 3D Medical Image Analysis with Variable Dimension Transform based Supervised 3D Pre-training](https://mlsanity.com/paper/2201.01426/) (2022)
- [Augmentation based unsupervised domain adaptation](https://mlsanity.com/paper/2202.11486/) (2022)
- [Integrative Imaging Informatics for Cancer Research: Workflow Automation for Neuro-oncology (I3CR-WANO)](https://mlsanity.com/paper/2210.03151/) (2022)
- [KonfAI: A Modular and Fully Configurable Framework for Deep Learning in Medical Imaging](https://mlsanity.com/paper/2508.09823/) (2025)
- [Artificial Intelligence in Tumor Subregion Analysis Based on Medical Imaging: A Review](https://mlsanity.com/paper/2103.13588/) (2021)
- [Generation of microbial colonies dataset with deep learning style transfer](https://mlsanity.com/paper/2111.03789/) (2021)
- [Kidney abnormality segmentation in thorax-abdomen CT scans](https://mlsanity.com/paper/2309.03383/) (2023)
- [MIP Candy: A Modular PyTorch Framework for Medical Image Processing](https://mlsanity.com/paper/2602.21033/) (2026)
- [MRI Images, Brain Lesions and Deep Learning](https://mlsanity.com/paper/2101.05091/) (2021)
- [An overview of open source Deep Learning-based libraries for Neuroscience](https://mlsanity.com/paper/2301.05057/) (2022)
- [Regional Deep Atrophy: a Self-Supervised Learning Method to Automatically Identify Regions Associated With Alzheimer's Disease Progression From Longitudinal MRI](https://mlsanity.com/paper/2304.04673/) (2023)

## Abstract (arXiv)

Medical image analysis and computer-assisted intervention problems are increasingly being addressed with deep-learning-based solutions. Established deep-learning platforms are flexible but do not provide specific functionality for medical image analysis and adapting them for this application requires substantial implementation effort. Thus, there has been substantial duplication of effort and incompatible infrastructure developed across many research groups. This work presents the open-source NiftyNet platform for deep learning in medical imaging. The ambition of NiftyNet is to accelerate and simplify the development of these solutions, and to provide a common mechanism for disseminating research outputs for the community to use, adapt and build upon. NiftyNet provides a modular deep-learning pipeline for a range of medical imaging applications including segmentation, regression, image generation and representation learning applications. Components of the NiftyNet pipeline including data loading, data augmentation, network architectures, loss functions and evaluation metrics are tailored to, and take advantage of, the idiosyncracies of medical image analysis and computer-assisted intervention. NiftyNet is built on TensorFlow and supports TensorBoard visualization of 2D and 3D images and computational graphs by default. We present 3 illustrative medical image analysis applications built using NiftyNet: (1) segmentation of multiple abdominal organs from computed tomography; (2) image regression to predict computed tomography attenuation maps from brain magnetic resonance images; and (3) generation of simulated ultrasound images for specified anatomical poses. NiftyNet enables researchers to rapidly develop and distribute deep learning solutions for segmentation, regression, image generation and representation learning applications, or extend the platform to new applications.