# Diversity by Design: Addressing Mode Collapse Improves scRNA-seq Perturbation Modeling on Well-Calibrated Metrics

arXiv: https://arxiv.org/abs/2506.22641 · HTML: https://mlsanity.com/paper/2506.22641/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 25 references that are in the map.
- First posted on arXiv 27 Jun 2025.

## Builds on (most cited first, 3 of 3)

- [PerturBench: Benchmarking Machine Learning Models for Cellular Perturbation Analysis](https://mlsanity.com/paper/2408.10609/) (2024)
- [Benchmarking Transcriptomics Foundation Models for Perturbation Analysis : one PCA still rules them all](https://mlsanity.com/paper/2410.13956/) (2024)
- [xTrimoGene: An Efficient and Scalable Representation Learner for Single-Cell RNA-Seq Data](https://mlsanity.com/paper/2311.15156/) (2023)

## Built on by (most cited first, 7 of 7)

- [scDFM: Distributional Flow Matching Model for Robust Single-Cell Perturbation Prediction](https://mlsanity.com/paper/2602.07103/) (2026)
- [Position: Biology is the Challenge Physics-Informed ML Needs to Evolve](https://mlsanity.com/paper/2510.25368/) (2025)
- [The Metric Picks the Winner: Evaluation Choice Flips Model Rankings for Drug-Response Prediction in Unseen Chemistry](https://mlsanity.com/paper/2606.12639/) (2026)
- [SCALE:Scalable Conditional Atlas-Level Endpoint transport for virtual cell perturbation prediction](https://mlsanity.com/paper/2603.17380/) (2026)
- [Can We Trust In-Distribution Success? Locked Evaluation Reveals Transfer Failure and Sampling-Depth Entanglement in CRISPRi Perturbation Prediction](https://mlsanity.com/paper/2608.00152/) (2026)
- [PopPert: Population-level Joint-Distribution Modeling for Single-Cell Perturbation Prediction](https://mlsanity.com/paper/2609.01357/) (2026)
- [Null importance: Disentangling relevance for interpretable machine learning](https://mlsanity.com/paper/2609.19511/) (2026)

## Abstract (arXiv)

Recent benchmarks reveal that models for single-cell perturbation response are often outperformed by simply predicting the dataset mean. We trace this anomaly to a metric artifact: control-referenced deltas and unweighted error metrics reward mode collapse whenever the control is biased or the biological signal is sparse. Large-scale textitin silico simulations and analysis of two real-world perturbation datasets confirm that shared reference shifts, not genuine biological change, drives high performance in these evaluations. We introduce differentially expressed gene (DEG)-aware metrics, weighted mean-squared error (WMSE) and weighted delta R2 (R2w(Delta)) with respect to all perturbations, that measure error in niche signals with high sensitivity. We further introduce negative and positive performance baselines to calibrate these metrics. With these improvements, the mean baseline sinks to null performance while genuine predictors are correctly rewarded. Finally, we show that using WMSE as a loss function reduces mode collapse and improves model performance.