# Retina U-Net: Embarrassingly Simple Exploitation of Segmentation Supervision for Medical Object Detection

arXiv: https://arxiv.org/abs/1811.08661 · HTML: https://mlsanity.com/paper/1811.08661/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Nov 2018.

## Built on by (most cited first, 30 of 47)

- [YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors](https://mlsanity.com/paper/2207.02696/) (2022)
- [Pix2seq: A Language Modeling Framework for Object Detection](https://mlsanity.com/paper/2109.10852/) (2021)
- [nnDetection: A Self-configuring Method for Medical Object Detection](https://mlsanity.com/paper/2106.00817/) (2021)
- [Common Limitations of Image Processing Metrics: A Picture Story](https://mlsanity.com/paper/2104.05642/) (2021)
- [DeGPR: Deep Guided Posterior Regularization for Multi-Class Cell Detection and Counting](https://mlsanity.com/paper/2304.00741/) (2023)
- [Identifying Label Errors in Object Detection Datasets by Loss Inspection](https://mlsanity.com/paper/2303.06999/) (2023)
- [Deep Learning for fully automatic detection, segmentation, and Gleason Grade estimation of prostate cancer in multiparametric Magnetic Resonance Images](https://mlsanity.com/paper/2103.12650/) (2021)
- [Unsupervised Contrastive Learning based Transformer for Lung Nodule Detection](https://mlsanity.com/paper/2205.00122/) (2022)
- [Longitudinal Segmentation of MS Lesions via Temporal Difference Weighting](https://mlsanity.com/paper/2409.13416/) (2024)
- [Potato Crop Stress Identification in Aerial Images using Deep Learning-based Object Detection](https://mlsanity.com/paper/2106.07770/) (2021)
- [Focused Decoding Enables 3D Anatomical Detection by Transformers](https://mlsanity.com/paper/2207.10774/) (2022)
- [Taming Detection Transformers for Medical Object Detection](https://mlsanity.com/paper/2306.15472/) (2023)
- [SATAY: A Streaming Architecture Toolflow for Accelerating YOLO Models on FPGA Devices](https://mlsanity.com/paper/2309.01587/) (2023)
- [Voxel Scene Graph for Intracranial Hemorrhage](https://mlsanity.com/paper/2407.21580/) (2024)
- [Artificial and beneficial -- Exploiting artificial images for aerial vehicle detection](https://mlsanity.com/paper/2104.03054/) (2021)
- [Robust Detection Outcome: A Metric for Pathology Detection in Medical Images](https://mlsanity.com/paper/2303.01920/) (2023)
- [YOLO-MED : Multi-Task Interaction Network for Biomedical Images](https://mlsanity.com/paper/2403.00245/) (2024)
- [Improving Segmentation and Detection of Lesions in CT Scans Using Intensity Distribution Supervision](https://mlsanity.com/paper/2307.05804/) (2023)
- [Detection of Intracranial Hemorrhage for Trauma Patients](https://mlsanity.com/paper/2408.10768/) (2024)
- [Self Pre-training with Adaptive Mask Autoencoders for Variable-Contrast 3D Medical Imaging](https://mlsanity.com/paper/2501.09096/) (2025)
- [The Application of Deep Learning for Lymph Node Segmentation: A Systematic Review](https://mlsanity.com/paper/2505.06118/) (2025)
- [The Missing Piece: A Case for Pre-Training in 3D Medical Object Detection](https://mlsanity.com/paper/2509.15947/) (2025)
- [Not Color Blind: AI Predicts Racial Identity from Black and White Retinal Vessel Segmentations](https://mlsanity.com/paper/2109.13845/) (2021)
- [Segmentation-Consistent Probabilistic Lesion Counting](https://mlsanity.com/paper/2204.05276/) (2022)
- [Z-SSMNet: Zonal-aware Self-supervised Mesh Network for Prostate Cancer Detection and Diagnosis with Bi-parametric MRI](https://mlsanity.com/paper/2212.05808/) (2022)
- [Accurate Detection of Mediastinal Lesions with nnDetection](https://mlsanity.com/paper/2303.11214/) (2023)
- [BugNIST -- a Large Volumetric Dataset for Object Detection under Domain Shift](https://mlsanity.com/paper/2304.01838/) (2023)
- [nnDetection for Intracranial Aneurysms Detection and Localization](https://mlsanity.com/paper/2305.13398/) (2023)
- [Weighted Circle Fusion: Ensembling Circle Representation from Different Object Detection Results](https://mlsanity.com/paper/2406.19540/) (2024)
- [FA-YOLO: Research On Efficient Feature Selection YOLO Improved Algorithm Based On FMDS and AGMF Modules](https://mlsanity.com/paper/2408.16313/) (2024)

## Abstract (arXiv)

The task of localizing and categorizing objects in medical images often remains formulated as a semantic segmentation problem. This approach, however, only indirectly solves the coarse localization task by predicting pixel-level scores, requiring ad-hoc heuristics when mapping back to object-level scores. State-of-the-art object detectors on the other hand, allow for individual object scoring in an end-to-end fashion, while ironically trading in the ability to exploit the full pixel-wise supervision signal. This can be particularly disadvantageous in the setting of medical image analysis, where data sets are notoriously small. In this paper, we propose Retina U-Net, a simple architecture, which naturally fuses the Retina Net one-stage detector with the U-Net architecture widely used for semantic segmentation in medical images. The proposed architecture recaptures discarded supervision signals by complementing object detection with an auxiliary task in the form of semantic segmentation without introducing the additional complexity of previously proposed two-stage detectors. We evaluate the importance of full segmentation supervision on two medical data sets, provide an in-depth analysis on a series of toy experiments and show how the corresponding performance gain grows in the limit of small data sets. Retina U-Net yields strong detection performance only reached by its more complex two-staged counterparts. Our framework including all methods implemented for operation on 2D and 3D images is available at github.com/pfjaeger/medicaldetectiontoolkit.