# Prior Information Guided Regularized Deep Learning for Cell Nucleus Detection

arXiv: https://arxiv.org/abs/1901.07061 · HTML: https://mlsanity.com/paper/1901.07061/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jan 2019.

## Built on by (most cited first, 6 of 6)

- [Deep learning for unsupervised domain adaptation in medical imaging: Recent advancements and future perspectives](https://mlsanity.com/paper/2308.01265/) (2023)
- [DeGPR: Deep Guided Posterior Regularization for Multi-Class Cell Detection and Counting](https://mlsanity.com/paper/2304.00741/) (2023)
- [Exploring Unsupervised Cell Recognition with Prior Self-activation Maps](https://mlsanity.com/paper/2308.11144/) (2023)
- [Unsupervised Dense Nuclei Detection and Segmentation with Prior Self-activation Map For Histology Images](https://mlsanity.com/paper/2210.07862/) (2022)
- [FourierLoss: Shape-Aware Loss Function with Fourier Descriptors](https://mlsanity.com/paper/2309.12106/) (2023)
- [Towards White-Box Deep Wireless Sensing](https://mlsanity.com/paper/2507.21799/) (2025)

## Abstract (arXiv)

Cell nuclei detection is a challenging research topic because of limitations in cellular image quality and diversity of nuclear morphology, i.e. varying nuclei shapes, sizes, and overlaps between multiple cell nuclei. This has been a topic of enduring interest with promising recent success shown by deep learning methods. These methods train Convolutional Neural Networks (CNNs) with a training set of input images and known, labeled nuclei locations. Many such methods are supplemented by spatial or morphological processing. Using a set of canonical cell nuclei shapes, prepared with the help of a domain expert, we develop a new approach that we call Shape Priors with Convolutional Neural Networks (SP-CNN). We further extend the network to introduce a shape prior (SP) layer and then allowing it to become trainable (i.e. optimizable). We call this network tunable SP-CNN (TSP-CNN). In summary, we present new network structures that can incorporate 'expected behavior' of nucleus shapes via two components: learnable layers that perform the nucleus detection and a fixed processing part that guides the learning with prior information. Analytically, we formulate two new regularization terms that are targeted at: 1) learning the shapes, 2) reducing false positives while simultaneously encouraging detection inside the cell nucleus boundary. Experimental results on two challenging datasets reveal that the proposed SP-CNN and TSP-CNN can outperform state-of-the-art alternatives.