# LangCell: Language-Cell Pre-training for Cell Identity Understanding

arXiv: https://arxiv.org/abs/2405.06708 · HTML: https://mlsanity.com/paper/2405.06708/

## Key facts

- Cited by 14 papers in the map (data through 7 Oct 2026).
- Builds on 15 of its 58 references that are in the map.
- First posted on arXiv 9 May 2024.

## Builds on (most cited first, 15 of 15)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigm](https://mlsanity.com/paper/2110.05208/) (2021)
- [Translation between Molecules and Natural Language](https://mlsanity.com/paper/2204.11817/) (2022)
- [What Do Self-Supervised Vision Transformers Learn?](https://mlsanity.com/paper/2305.00729/) (2023)
- [ProtST: Multi-Modality Learning of Protein Sequences and Biomedical Texts](https://mlsanity.com/paper/2301.12040/) (2023)
- [xTrimoGene: An Efficient and Scalable Representation Learner for Single-Cell RNA-Seq Data](https://mlsanity.com/paper/2311.15156/) (2023)
- [Multi-modal Self-supervised Pre-training for Regulatory Genome Across Cell Types](https://mlsanity.com/paper/2110.05231/) (2021)
- [A single-cell gene expression language model](https://mlsanity.com/paper/2210.14330/) (2022)
- [Cell Identity Codes: Understanding Cell Identity from Gene Expression Profiles using Deep Neural Networks](https://mlsanity.com/paper/1806.04863/) (2018)

## Built on by (most cited first, 14 of 14)

- [Learning Multi-view Molecular Representations with Structured and Unstructured Knowledge](https://mlsanity.com/paper/2406.09841/) (2024)
- [SToFM: a Multi-scale Foundation Model for Spatial Transcriptomics](https://mlsanity.com/paper/2507.11588/) (2025)
- [Language-Enhanced Representation Learning for Single-Cell Transcriptomics](https://mlsanity.com/paper/2503.09427/) (2025)
- [GeneMamba: An Efficient and Effective Foundation Model on Single Cell Data](https://mlsanity.com/paper/2504.16956/) (2025)
- [Prototype Guided Post-pretraining for Single-Cell Representation Learning](https://mlsanity.com/paper/2605.07938/) (2026)
- [Spatial Coordinates as a Cell Language: A Multi-Sentence Framework for Imaging Mass Cytometry Analysis](https://mlsanity.com/paper/2506.01918/) (2025)
- [GATHER: Convergence-Centric Hyper-Entity Retrieval for Zero-Shot Cell-Type Annotation](https://mlsanity.com/paper/2605.06403/) (2026)
- [HDTree: Generative Modeling of Cellular Hierarchies for Robust Lineage Inference](https://mlsanity.com/paper/2506.23287/) (2025)
- [GRIT: Graph-Regularized Logit Refinement for Zero-shot Cell Type Annotation](https://mlsanity.com/paper/2508.04747/) (2025)
- [Causality-Induced Positional Encoding for Transformer-Based Representation Learning of Non-Sequential Features](https://mlsanity.com/paper/2509.16629/) (2025)
- [BioVERSE: Representation Alignment of Biomedical Modalities to LLMs for Multi-Modal Reasoning](https://mlsanity.com/paper/2510.01428/) (2025)
- [Open World Knowledge Aided Single-Cell Foundation Model with Robust Cross-Modal Cell-Language Pre-training](https://mlsanity.com/paper/2601.05648/) (2026)
- [Towards a knowledge-enhanced single-cell foundation model](https://mlsanity.com/paper/2609.14970/) (2026)
- [CellMSA: Context Modeling for Single-Cell Representation Learning](https://mlsanity.com/paper/2609.38908/) (2026)

## Abstract (arXiv)

Cell identity encompasses various semantic aspects of a cell, including cell type, pathway information, disease information, and more, which are essential for biologists to gain insights into its biological characteristics. Understanding cell identity from the transcriptomic data, such as annotating cell types, has become an important task in bioinformatics. As these semantic aspects are determined by human experts, it is impossible for AI models to effectively carry out cell identity understanding tasks without the supervision signals provided by single-cell and label pairs. The single-cell pre-trained language models (PLMs) currently used for this task are trained only on a single modality, transcriptomics data, lack an understanding of cell identity knowledge. As a result, they have to be fine-tuned for downstream tasks and struggle when lacking labeled data with the desired semantic labels. To address this issue, we propose an innovative solution by constructing a unified representation of single-cell data and natural language during the pre-training phase, allowing the model to directly incorporate insights related to cell identity. More specifically, we introduce LangCell, the first Language-Cell pre-training framework. LangCell utilizes texts enriched with cell identity information to gain a profound comprehension of cross-modal knowledge. Results from experiments conducted on different benchmarks show that LangCell is the only single-cell PLM that can work effectively in zero-shot cell identity understanding scenarios, and also significantly outperforms existing models in few-shot and fine-tuning cell identity understanding scenarios.