# Multi-omics Prediction from High-content Cellular Imaging with Deep Learning

arXiv: https://arxiv.org/abs/2306.09391 · HTML: https://mlsanity.com/paper/2306.09391/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 60 references that are in the map.
- First posted on arXiv 15 Jun 2023.

## Builds on (most cited first, 10 of 10)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Return of Frustratingly Easy Domain Adaptation](https://mlsanity.com/paper/1511.05547/) (2015)
- [Learning Neural Causal Models with Active Interventions](https://mlsanity.com/paper/2109.02429/) (2021)
- [Overcoming Barriers to Data Sharing with Medical Image Generation: A Comprehensive Evaluation](https://mlsanity.com/paper/2012.03769/) (2020)
- [CausalBench: A Large-scale Benchmark for Network Inference from Single-cell Perturbation Data](https://mlsanity.com/paper/2210.17283/) (2022)
- [GeneDisco: A Benchmark for Experimental Design in Drug Discovery](https://mlsanity.com/paper/2110.11875/) (2021)
- [CytoImageNet: A large-scale pretraining dataset for bioimage transfer learning](https://mlsanity.com/paper/2111.11646/) (2021)
- [Genotype-Guided Radiomics Signatures for Recurrence Prediction of Non-Small-Cell Lung Cancer](https://mlsanity.com/paper/2104.14420/) (2021)

## Built on by (most cited first, 3 of 3)

- [In-silico biological discovery with large perturbation models](https://mlsanity.com/paper/2503.23535/) (2025)
- [URLOST: Unsupervised Representation Learning without Stationarity or Topology](https://mlsanity.com/paper/2310.04496/) (2023)
- [Out-of-distribution evaluations of channel agnostic masked autoencoders in fluorescence microscopy](https://mlsanity.com/paper/2503.19149/) (2025)

## Abstract (arXiv)

High-content cellular imaging, transcriptomics, and proteomics data provide rich and complementary views on the molecular layers of biology that influence cellular states and function. However, the biological determinants through which changes in multi-omics measurements influence cellular morphology have not yet been systematically explored, and the degree to which cell imaging could potentially enable the prediction of multi-omics directly from cell imaging data is therefore currently unclear. Here, we address the question of whether it is possible to predict bulk multi-omics measurements directly from cell images using Image2Omics - a deep learning approach that predicts multi-omics in a cell population directly from high-content images of cells stained with multiplexed fluorescent dyes. We perform an experimental evaluation in gene-edited macrophages derived from human induced pluripotent stem cells (hiPSC) under multiple stimulation conditions and demonstrate that Image2Omics achieves significantly better performance in predicting transcriptomics and proteomics measurements directly from cell images than predictions based on the mean observed training set abundance. We observed significant predictability of abundances for 4927 (18.72%; 95% CI: 6.52%, 35.52%) and 3521 (13.38%; 95% CI: 4.10%, 32.21%) transcripts out of 26137 in M1 and M2-stimulated macrophages respectively and for 422 (8.46%; 95% CI: 0.58%, 25.83%) and 697 (13.98%; 95% CI: 2.41%, 32.83%) proteins out of 4986 in M1 and M2-stimulated macrophages respectively. Our results show that some transcript and protein abundances are predictable from cell imaging and that cell imaging may potentially, in some settings and depending on the mechanisms of interest and desired performance threshold, even be a scalable and resource-efficient substitute for multi-omics measurements.