# GenCellAgent: Generalizable, Training-Free Cellular Image Segmentation via Large Language Model Agents

arXiv: https://arxiv.org/abs/2510.13896 · HTML: https://mlsanity.com/paper/2510.13896/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 82 references that are in the map.
- First posted on arXiv 14 Oct 2025.

## Builds on (most cited first, 21 of 21)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [LISA: Reasoning Segmentation via Large Language Model](https://mlsanity.com/paper/2308.00692/) (2023)
- [Large Language Model based Multi-Agents: A Survey of Progress and Challenges](https://mlsanity.com/paper/2402.01680/) (2024)
- [A Survey on the Memory Mechanism of Large Language Model based Agents](https://mlsanity.com/paper/2404.13501/) (2024)
- [Image Segmentation Using Text and Image Prompts](https://mlsanity.com/paper/2112.10003/) (2021)
- [Agent Laboratory: Using LLM Agents as Research Assistants](https://mlsanity.com/paper/2501.04227/) (2025)
- [Understanding the planning of LLM agents: A survey](https://mlsanity.com/paper/2402.02716/) (2024)
- [Large Language Model Agent: A Survey on Methodology, Applications and Challenges](https://mlsanity.com/paper/2503.21460/) (2025)
- [SegGPT: Segmenting Everything In Context](https://mlsanity.com/paper/2304.03284/) (2023)
- [From Automation to Autonomy: A Survey on Large Language Models in Scientific Discovery](https://mlsanity.com/paper/2505.13259/) (2025)
- [Lizard: A Large-Scale Dataset for Colonic Nuclear Instance Segmentation and Classification](https://mlsanity.com/paper/2108.11195/) (2021)
- [Agentic AI for Scientific Discovery: A Survey of Progress, Challenges, and Future Directions](https://mlsanity.com/paper/2503.08979/) (2025)
- [CRISPR-GPT for Agentic Automation of Gene-editing Experiments](https://mlsanity.com/paper/2404.18021/) (2024)
- [Autonomous LLM-driven research from data to human-verifiable research papers](https://mlsanity.com/paper/2404.17605/) (2024)
- [CellAgent: An LLM-driven Multi-Agent Framework for Automated Single-cell Data Analysis](https://mlsanity.com/paper/2407.09811/) (2024)
- [A Vision for Auto Research with LLM Agents](https://mlsanity.com/paper/2504.18765/) (2025)
- [CSR-Bench: Benchmarking LLM Agents in Deployment of Computer Science Research Repositories](https://mlsanity.com/paper/2502.06111/) (2025)
- [BioImage.IO Chatbot: A Community-Driven AI Assistant for Integrative Computational Bioimaging](https://mlsanity.com/paper/2310.18351/) (2023)
- [Nellie: Automated organelle segmentation, tracking, and hierarchical feature extraction in 2D/3D live-cell microscopy](https://mlsanity.com/paper/2403.13214/) (2024)
- [Automating Intervention Discovery from Scientific Literature: A Progressive Ontology Prompting and Dual-LLM Framework](https://mlsanity.com/paper/2409.00054/) (2024)

## Built on by (most cited first, 3 of 3)

- [AutoSizer: Automatic Sizing of Analog and Mixed-Signal Circuits via Large Language Model (LLM) Agents](https://mlsanity.com/paper/2602.02849/) (2026)
- [AblateCell: A Reproduce-then-Ablate Agent for Virtual Cell Repositories](https://mlsanity.com/paper/2604.19606/) (2026)
- [BIABench: Evaluating AI agents on real-world bioimage analysis tasks](https://mlsanity.com/paper/2609.34274/) (2026)

## Abstract (arXiv)

Cellular image segmentation is essential for quantitative biology yet remains difficult due to heterogeneous modalities, morphological variability, and limited annotations. We present GenCellAgent, a training-free multi-agent framework that orchestrates specialist segmenters and generalist vision-language models via a planner-executor-evaluator loop (choose tool rightarrow run rightarrow quality-check) with long-term memory. The system (i) automatically routes images to the best tool, (ii) adapts on the fly using a few reference images when imaging conditions differ from what a tool expects, (iii) supports text-guided segmentation of organelles not covered by existing models, and (iv) commits expert edits to memory, enabling self-evolution and personalized workflows. Across seven cell-segmentation benchmarks spanning diverse microscopy modalities (4,718 images), this routing consistently matches or exceeds the best individual tool on every dataset and outperforms all baselines in overall accuracy. On out-of-distribution organelle data, GenCellAgent substantially outperforms specialist models that were not trained on the target domain, recovering structures that dedicated tools fail to detect. It also segments novel objects such as the Golgi apparatus via iterative text-guided refinement, with light human correction further boosting performance. Together, these capabilities provide a practical path to robust, adaptable cellular image segmentation without retraining, while reducing annotation burden and matching user preferences.