# The Multi-modality Cell Segmentation Challenge: Towards Universal Solutions

arXiv: https://arxiv.org/abs/2308.05864 · HTML: https://mlsanity.com/paper/2308.05864/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 75 references that are in the map.
- First posted on arXiv 10 Aug 2023.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [HoVer-Net: Simultaneous Segmentation and Classification of Nuclei in Multi-Tissue Histology Images](https://mlsanity.com/paper/1812.06499/) (2018)
- [Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation](https://mlsanity.com/paper/1908.10454/) (2019)
- [Segment Anything in Medical Images](https://mlsanity.com/paper/2304.12306/) (2023)
- [Metrics reloaded: Recommendations for image analysis validation](https://mlsanity.com/paper/2206.01653/) (2022)
- [Cell Detection with Star-convex Polygons](https://mlsanity.com/paper/1806.03535/) (2018)
- [Using Hindsight to Anchor Past Knowledge in Continual Learning](https://mlsanity.com/paper/2002.08165/) (2020)
- [Why rankings of biomedical image analysis competitions should be interpreted with care](https://mlsanity.com/paper/1806.02051/) (2018)
- [Unbiased Teacher v2: Semi-supervised Object Detection for Anchor-free and Anchor-based Detectors](https://mlsanity.com/paper/2206.09500/) (2022)
- [Methods and open-source toolkit for analyzing and visualizing challenge results](https://mlsanity.com/paper/1910.05121/) (2019)
- [Multi-Task Attention-Based Semi-Supervised Learning for Medical Image Segmentation](https://mlsanity.com/paper/1907.12303/) (2019)
- [CoNIC Challenge: Pushing the Frontiers of Nuclear Detection, Segmentation, Classification and Counting](https://mlsanity.com/paper/2303.06274/) (2023)
- [Localization Uncertainty Estimation for Anchor-Free Object Detection](https://mlsanity.com/paper/2006.15607/) (2020)
- [Contour Proposal Networks for Biomedical Instance Segmentation](https://mlsanity.com/paper/2104.03393/) (2021)
- [MEDIAR: Harmony of Data-Centric and Model-Centric for Multi-Modality Microscopy](https://mlsanity.com/paper/2212.03465/) (2022)
- [J Regularization Improves Imbalanced Multiclass Segmentation](https://mlsanity.com/paper/1910.09783/) (2019)

## Built on by (most cited first, 30 of 39)

- [U-Mamba: Enhancing Long-range Dependency for Biomedical Image Segmentation](https://mlsanity.com/paper/2401.04722/) (2024)
- [Swin-UMamba: Mamba-based UNet with ImageNet-based pretraining](https://mlsanity.com/paper/2402.03302/) (2024)
- [MMedAgent: Learning to Use Medical Tools with Multi-modal Agent](https://mlsanity.com/paper/2407.02483/) (2024)
- [Vision Mamba: A Comprehensive Survey and Taxonomy](https://mlsanity.com/paper/2405.04404/) (2024)
- [Segment Anything in Medical Images and Videos: Benchmark and Deployment](https://mlsanity.com/paper/2408.03322/) (2024)
- [Biomedical SAM 2: Segment Anything in Biomedical Images and Videos](https://mlsanity.com/paper/2408.03286/) (2024)
- [xLSTM-UNet can be an Effective 2D & 3D Medical Image Segmentation Backbone with Vision-LSTM (ViL) better than its Mamba Counterpart](https://mlsanity.com/paper/2407.01530/) (2024)
- [ViM-UNet: Vision Mamba for Biomedical Segmentation](https://mlsanity.com/paper/2404.07705/) (2024)
- [Multimodal Data Integration for Precision Oncology: Challenges and Future Directions](https://mlsanity.com/paper/2406.19611/) (2024)
- [Cell Graph Transformer for Nuclei Classification](https://mlsanity.com/paper/2402.12946/) (2024)
- [LoG-VMamba: Local-Global Vision Mamba for Medical Image Segmentation](https://mlsanity.com/paper/2408.14415/) (2024)
- [HoloHisto: End-to-end Gigapixel WSI Segmentation with 4K Resolution Sequential Tokenization](https://mlsanity.com/paper/2407.03307/) (2024)
- [SAM3-Adapter: Efficient Adaptation of Segment Anything 3 for Camouflage Object Segmentation, Shadow Detection, and Medical Image Segmentation](https://mlsanity.com/paper/2511.19425/) (2025)
- [CellSeg1: Robust Cell Segmentation with One Training Image](https://mlsanity.com/paper/2412.01410/) (2024)
- [TTT-Unet: Enhancing U-Net with Test-Time Training Layers for Biomedical Image Segmentation](https://mlsanity.com/paper/2409.11299/) (2024)
- [Enhancing Feature Fusion of U-like Networks with Dynamic Skip Connections](https://mlsanity.com/paper/2509.14610/) (2025)
- [Defining the boundaries: challenges and advances in identifying cells in microscopy images](https://mlsanity.com/paper/2311.08269/) (2023)
- [UniCell: Universal Cell Nucleus Classification via Prompt Learning](https://mlsanity.com/paper/2402.12938/) (2024)
- [PyUAT: Open-source Python framework for efficient and scalable cell tracking](https://mlsanity.com/paper/2503.21914/) (2025)
- [Fast Sphericity and Roundness approximation in 2D and 3D using Local Thickness](https://mlsanity.com/paper/2504.05808/) (2025)
- [MedVKAN: Efficient Feature Extraction with Mamba and KAN for Medical Image Segmentation](https://mlsanity.com/paper/2505.11797/) (2025)
- [SEMA: a Scalable and Efficient Mamba like Attention via Token Localization and Averaging](https://mlsanity.com/paper/2506.08297/) (2025)
- [NOA: a versatile, extensible tool for AI-based organoid analysis](https://mlsanity.com/paper/2511.01549/) (2025)
- [CHOTA: A Higher Order Accuracy Metric for Cell Tracking](https://mlsanity.com/paper/2408.11571/) (2024)
- [CellPilot: A unified approach to automatic and interactive segmentation in histopathology](https://mlsanity.com/paper/2411.15514/) (2024)
- [From Claims to Evidence: A Unified Framework and Critical Analysis of CNN vs. Transformer vs. Mamba in Medical Image Segmentation](https://mlsanity.com/paper/2503.01306/) (2025)
- [SAMJ: Fast Image Annotation on ImageJ/Fiji via Segment Anything Model](https://mlsanity.com/paper/2506.02783/) (2025)
- [Data Efficiency and Transfer Robustness in Biomedical Image Segmentation: A Study of Redundancy and Forgetting with Cellpose](https://mlsanity.com/paper/2511.04803/) (2025)
- [Simple Agents Outperform Experts in Biomedical Imaging Workflow Optimization](https://mlsanity.com/paper/2512.06006/) (2025)
- [Cell Behavior Video Classification Challenge, a benchmark for computer vision methods in time-lapse microscopy](https://mlsanity.com/paper/2601.10250/) (2026)

## Abstract (arXiv)

Cell segmentation is a critical step for quantitative single-cell analysis in microscopy images. Existing cell segmentation methods are often tailored to specific modalities or require manual interventions to specify hyper-parameters in different experimental settings. Here, we present a multi-modality cell segmentation benchmark, comprising over 1500 labeled images derived from more than 50 diverse biological experiments. The top participants developed a Transformer-based deep-learning algorithm that not only exceeds existing methods but can also be applied to diverse microscopy images across imaging platforms and tissue types without manual parameter adjustments. This benchmark and the improved algorithm offer promising avenues for more accurate and versatile cell analysis in microscopy imaging.