One Click per Cell Type Suffices: Training-free Group Interaction for Cell Instance Segmentation

Cell instance segmentation models trained on cell-specific datasets suffer severe performance drops on out-of-distribution cell types, while interactive foundation models overcome this through per-instance prompting at a cost that is prohibitively expensive for histopathology images containing hundreds to thousands of densely packed instances. We introduce Group Prompting, a new paradigm that shifts interactive segmentation from per-instance O(N) to per-type O(T), where a single click per cell type suffices to segment all instances of that type. Our key observation is that the frozen image encoder of the Segment Anything Model (SAM) already clusters same-type cells in its feature space before any prompt is given, and that this clustering holds across staining modalities without any training. Exploiting this property, we propose Chain-of-Prompts (CoP), a training-free framework that recursively expands a single user click by (1) identifying reliable same-type locations through non-parametric gating of multi-scale encoder features, and (2) selecting the most spatially distant reliable point as the next prompt to maximize coverage. On eleven benchmarks, CoP generalizes to both unseen cell types and unseen imaging modalities without any adaptation: with one click per type it retains over 90% of per-instance performance on three cell-type-annotated datasets while surpassing fully-supervised methods, and with one click per image it retains over 95% on eight datasets spanning both H&E and non-H&E imaging. Project Page: https://shjo-april.github.io/Chain-of-Prompts/