# Automatically Designing CNN Architectures for Medical Image Segmentation

arXiv: https://arxiv.org/abs/1807.07663 · HTML: https://mlsanity.com/paper/1807.07663/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jul 2018.

## Built on by (most cited first, 8 of 8)

- [A Survey of Geometric Optimization for Deep Learning: From Euclidean Space to Riemannian Manifold](https://mlsanity.com/paper/2302.08210/) (2023)
- [DQNAS: Neural Architecture Search using Reinforcement Learning](https://mlsanity.com/paper/2301.06687/) (2023)
- [DinoAtten3D: Slice-Level Attention Aggregation of DinoV2 for 3D Brain MRI Anomaly Classification](https://mlsanity.com/paper/2509.12512/) (2025)
- [Direct evaluation of progression or regression of disease burden in brain metastatic disease with Deep Neuroevolution](https://mlsanity.com/paper/2203.12853/) (2022)
- [The Past, Current, and Future of Neonatal Intensive Care Units with Artificial Intelligence](https://mlsanity.com/paper/2302.00225/) (2023)
- [Implantable Adaptive Cells: A Novel Enhancement for Pre-Trained U-Nets in Medical Image Segmentation](https://mlsanity.com/paper/2405.03420/) (2024)
- [S-E Pipeline: A Vision Transformer (ViT) based Resilient Classification Pipeline for Medical Imaging Against Adversarial Attacks](https://mlsanity.com/paper/2407.17587/) (2024)
- [3D Segment Anything Model with Visual Mamba for Diagnosing Placenta Accreta Spectrum](https://mlsanity.com/paper/2606.00489/) (2026)

## Abstract (arXiv)

Deep neural network architectures have traditionally been designed and explored with human expertise in a long-lasting trial-and-error process. This process requires huge amount of time, expertise, and resources. To address this tedious problem, we propose a novel algorithm to optimally find hyperparameters of a deep network architecture automatically. We specifically focus on designing neural architectures for medical image segmentation task. Our proposed method is based on a policy gradient reinforcement learning for which the reward function is assigned a segmentation evaluation utility (i.e., dice index). We show the efficacy of the proposed method with its low computational cost in comparison with the state-of-the-art medical image segmentation networks. We also present a new architecture design, a densely connected encoder-decoder CNN, as a strong baseline architecture to apply the proposed hyperparameter search algorithm. We apply the proposed algorithm to each layer of the baseline architectures. As an application, we train the proposed system on cine cardiac MR images from Automated Cardiac Diagnosis Challenge (ACDC) MICCAI 2017. Starting from a baseline segmentation architecture, the resulting network architecture obtains the state-of-the-art results in accuracy without performing any trial-and-error based architecture design approaches or close supervision of the hyperparameters changes.