# IRNet: Instance Relation Network for Overlapping Cervical Cell Segmentation

arXiv: https://arxiv.org/abs/1908.06623 · HTML: https://mlsanity.com/paper/1908.06623/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Aug 2019.

## Built on by (most cited first, 15 of 15)

- [Deep Learning for Computational Cytology: A Survey](https://mlsanity.com/paper/2202.05126/) (2022)
- [InsMix: Towards Realistic Generative Data Augmentation for Nuclei Instance Segmentation](https://mlsanity.com/paper/2206.15134/) (2022)
- [DoNet: Deep De-overlapping Network for Cytology Instance Segmentation](https://mlsanity.com/paper/2303.14373/) (2023)
- [Semantic-aware SAM for Point-Prompted Instance Segmentation](https://mlsanity.com/paper/2312.15895/) (2023)
- [Deep Learning in Breast Cancer Imaging: A Decade of Progress and Future Directions](https://mlsanity.com/paper/2304.06662/) (2023)
- [BoNuS: Boundary Mining for Nuclei Segmentation with Partial Point Labels](https://mlsanity.com/paper/2401.07437/) (2024)
- [What is the State of the Art of Computer Vision-Assisted Cytology? A Systematic Literature Review](https://mlsanity.com/paper/2105.11277/) (2021)
- [GAInS: Gradient Anomaly-aware Biomedical Instance Segmentation](https://mlsanity.com/paper/2409.13988/) (2024)
- [Exploring Contextual Relationships for Cervical Abnormal Cell Detection](https://mlsanity.com/paper/2207.04693/) (2022)
- [AMA-SAM: Adversarial Multi-Domain Alignment of Segment Anything Model for High-Fidelity Histology Nuclei Segmentation](https://mlsanity.com/paper/2503.21695/) (2025)
- [P2Object: Single Point Supervised Object Detection and Instance Segmentation](https://mlsanity.com/paper/2504.07813/) (2025)
- [Skeleton-Guided Sparse Anchors for Rotated Instance Segmentation in Cell Microscopy](https://mlsanity.com/paper/2401.09895/) (2024)
- [A Survey on Cell Nuclei Instance Segmentation and Classification: Leveraging Context and Attention](https://mlsanity.com/paper/2407.18673/) (2024)
- [Weak-to-Strong Generalization Enables Fully Automated De Novo Training of Multi-head Mask-RCNN Model for Segmenting Densely Overlapping Cell Nuclei in Multiplex Whole-slice Brain Images](https://mlsanity.com/paper/2512.11722/) (2025)
- [SAPNet++: Evolving Point-Prompted Instance Segmentation with Semantic and Spatial Awareness](https://mlsanity.com/paper/2602.21762/) (2026)

## Abstract (arXiv)

Cell instance segmentation in Pap smear image remains challenging due to the wide existence of occlusion among translucent cytoplasm in cell clumps. Conventional methods heavily rely on accurate nuclei detection results and are easily disturbed by miscellaneous objects. In this paper, we propose a novel Instance Relation Network (IRNet) for robust overlapping cell segmentation by exploring instance relation interaction. Specifically, we propose the Instance Relation Module to construct the cell association matrix for transferring information among individual cell-instance features. With the collaboration of different instances, the augmented features gain benefits from contextual information and improve semantic consistency. Meanwhile, we proposed a sparsity constrained Duplicate Removal Module to eliminate the misalignment between classification and localization accuracy for candidates selection. The largest cervical Pap smear (CPS) dataset with more than 8000 cell annotations in Pap smear image was constructed for comprehensive evaluation. Our method outperforms other methods by a large margin, demonstrating the effectiveness of exploring instance relation.