# A Multi-Modal AI Copilot for Single-Cell Analysis with Instruction Following

arXiv: https://arxiv.org/abs/2501.08187 · HTML: https://mlsanity.com/paper/2501.08187/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jan 2025.

## Built on by (most cited first, 4 of 4)

- [Cell-o1: Training LLMs to Solve Single-Cell Reasoning Puzzles with Reinforcement Learning](https://mlsanity.com/paper/2506.02911/) (2025)
- [Language-Enhanced Representation Learning for Single-Cell Transcriptomics](https://mlsanity.com/paper/2503.09427/) (2025)
- [Large Language Models Meet Virtual Cell: A Survey](https://mlsanity.com/paper/2510.07706/) (2025)
- [ES-Merging: Biological MLLM Merging via Embedding Space Signals](https://mlsanity.com/paper/2603.14405/) (2026)

## Abstract (arXiv)

Large language models excel at interpreting complex natural language instructions, enabling them to perform a wide range of tasks. In the life sciences, single-cell RNA sequencing (scRNA-seq) data serves as the "language of cellular biology", capturing intricate gene expression patterns at the single-cell level. However, interacting with this "language" through conventional tools is often inefficient and unintuitive, posing challenges for researchers. To address these limitations, we present InstructCell, a multi-modal AI copilot that leverages natural language as a medium for more direct and flexible single-cell analysis. We construct a comprehensive multi-modal instruction dataset that pairs text-based instructions with scRNA-seq profiles from diverse tissues and species. Building on this, we develop a multi-modal cell language architecture capable of simultaneously interpreting and processing both modalities. InstructCell empowers researchers to accomplish critical tasks-such as cell type annotation, conditional pseudo-cell generation, and drug sensitivity prediction-using straightforward natural language commands. Extensive evaluations demonstrate that InstructCell consistently meets or exceeds the performance of existing single-cell foundation models, while adapting to diverse experimental conditions. More importantly, InstructCell provides an accessible and intuitive tool for exploring complex single-cell data, lowering technical barriers and enabling deeper biological insights.