# Benchmarking Transcriptomics Foundation Models for Perturbation Analysis : one PCA still rules them all

arXiv: https://arxiv.org/abs/2410.13956 · HTML: https://mlsanity.com/paper/2410.13956/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 43 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Builds on (most cited first, 4 of 4)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [DiffDock: Diffusion Steps, Twists, and Turns for Molecular Docking](https://mlsanity.com/paper/2210.01776/) (2022)
- [Masked Autoencoders for Microscopy are Scalable Learners of Cellular Biology](https://mlsanity.com/paper/2404.10242/) (2024)
- [Weakly Supervised Set-Consistency Learning Improves Morphological Profiling of Single-Cell Images](https://mlsanity.com/paper/2406.05308/) (2024)

## Built on by (most cited first, 19 of 19)

- [CellForge: Agentic Design of Virtual Cell Models](https://mlsanity.com/paper/2508.02276/) (2025)
- [TxPert: Leveraging Biochemical Relationships for Out-of-Distribution Transcriptomic Perturbation Prediction](https://mlsanity.com/paper/2505.14919/) (2025)
- [Virtual Cells: Predict, Explain, Discover](https://mlsanity.com/paper/2505.14613/) (2025)
- [Diversity by Design: Addressing Mode Collapse Improves scRNA-seq Perturbation Modeling on Well-Calibrated Metrics](https://mlsanity.com/paper/2506.22641/) (2025)
- [JoyAgent-JDGenie: Technical Report on the GAIA](https://mlsanity.com/paper/2510.00510/) (2025)
- [What Makes a Representation Good for Single-Cell Perturbation Prediction?](https://mlsanity.com/paper/2605.19343/) (2026)
- [No Foundations without Foundations -- Why semi-mechanistic models are essential for regulatory biology](https://mlsanity.com/paper/2501.19178/) (2025)
- [A Cross Modal Knowledge Distillation & Data Augmentation Recipe for Improving Transcriptomics Representations through Morphological Features](https://mlsanity.com/paper/2505.21317/) (2025)
- [Adaptive Data-Knowledge Alignment in Genetic Perturbation Prediction](https://mlsanity.com/paper/2510.00512/) (2025)
- [Gamma Mixture Modeling for Cosine Similarity in Small Language Models](https://mlsanity.com/paper/2510.05309/) (2025)
- [Multi-context principal component analysis](https://mlsanity.com/paper/2601.15239/) (2026)
- [Deep Learning for BioImaging: What Are We Really Learning?](https://mlsanity.com/paper/2603.13377/) (2026)
- [Effective Biological Representation Learning by Masking Gene Expression](https://mlsanity.com/paper/2605.31562/) (2026)
- [Knowledge Graphs and Reasoning LLMs for Finding Simple Yet Effective Transcriptomic Perturbation Predictors](https://mlsanity.com/paper/2606.08816/) (2026)
- [Probing, Fusion, and Trustworthiness: A Systematic Evaluation of Foundation Model Representations for Multimodal Cancer Analysis](https://mlsanity.com/paper/2606.17115/) (2026)
- [Can We Trust In-Distribution Success? Locked Evaluation Reveals Transfer Failure and Sampling-Depth Entanglement in CRISPRi Perturbation Prediction](https://mlsanity.com/paper/2608.00152/) (2026)
- [Language as the Interface: Foundation-Model Contrastive Learning Links Transcriptomes and Electrophysiology](https://mlsanity.com/paper/2609.37024/) (2026)
- [Disentangling Marginal Capacity from Geometry in Optimal-Transport Label Transfer](https://mlsanity.com/paper/2610.14853/) (2026)
- [Do Perturbation Predictors Learn Gene Interactions? Diagnosing Extrapolation in Agentic Screening](https://mlsanity.com/paper/2610.18880/) (2026)

## Abstract (arXiv)

Understanding the relationships among genes, compounds, and their interactions in living organisms remains limited due to technological constraints and the complexity of biological data. Deep learning has shown promise in exploring these relationships using various data types. However, transcriptomics, which provides detailed insights into cellular states, is still underused due to its high noise levels and limited data availability. Recent advancements in transcriptomics sequencing provide new opportunities to uncover valuable insights, especially with the rise of many new foundation models for transcriptomics, yet no benchmark has been made to robustly evaluate the effectiveness of these rising models for perturbation analysis. This article presents a novel biologically motivated evaluation framework and a hierarchy of perturbation analysis tasks for comparing the performance of pretrained foundation models to each other and to more classical techniques of learning from transcriptomics data. We compile diverse public datasets from different sequencing techniques and cell lines to assess models performance. Our approach identifies scVI and PCA to be far better suited models for understanding biological perturbations in comparison to existing foundation models, especially in their application in real-world scenarios.