# STimage-1K4M: A histopathology image-gene expression dataset for spatial transcriptomics

arXiv: https://arxiv.org/abs/2406.06393 · HTML: https://mlsanity.com/paper/2406.06393/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 169 references that are in the map.
- First posted on arXiv 10 Jun 2024.

## Builds on (most cited first, 12 of 12)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning](https://mlsanity.com/paper/2103.01913/) (2021)
- [Quilt-1M: One Million Image-Text Pairs for Histopathology](https://mlsanity.com/paper/2306.11207/) (2023)
- [CellViT: Vision Transformers for Precise Cell Segmentation and Classification](https://mlsanity.com/paper/2306.15350/) (2023)
- [Lizard: A Large-Scale Dataset for Colonic Nuclear Instance Segmentation and Classification](https://mlsanity.com/paper/2108.11195/) (2021)
- [Spatially Resolved Gene Expression Prediction from H&E Histology Images via Bi-modal Contrastive Learning](https://mlsanity.com/paper/2306.01859/) (2023)
- [Multiple Instance Captioning: Learning Representations from Histopathology Textbooks and Articles](https://mlsanity.com/paper/2103.05121/) (2021)
- [Overcoming the limitations of patch-based learning to detect cancer in whole slide images](https://mlsanity.com/paper/2012.00617/) (2020)
- [Multi-modal Masked Autoencoders Learn Compositional Histopathological Representations](https://mlsanity.com/paper/2209.01534/) (2022)

## Built on by (most cited first, 23 of 23)

- [Scalable Generation of Spatial Transcriptomics from Histology Images via Whole-Slide Flow Matching](https://mlsanity.com/paper/2506.05361/) (2025)
- [SToFM: a Multi-scale Foundation Model for Spatial Transcriptomics](https://mlsanity.com/paper/2507.11588/) (2025)
- [Spatially Gene Expression Prediction using Dual-Scale Contrastive Learning](https://mlsanity.com/paper/2506.23827/) (2025)
- [From Classical Machine Learning to Emerging Foundation Models: Review on Multimodal Data Integration for Cancer Research](https://mlsanity.com/paper/2507.09028/) (2025)
- [Towards Unified Molecule-Enhanced Pathology Image Representation Learning via Integrating Spatial Transcriptomics](https://mlsanity.com/paper/2412.00651/) (2024)
- [TeamPath: Building MultiModal Pathology Experts with Reasoning AI Copilots](https://mlsanity.com/paper/2511.17652/) (2025)
- [HistoPrism: Unlocking Functional Pathway Analysis from Pan-Cancer Histology via Gene Expression Prediction](https://mlsanity.com/paper/2601.21560/) (2026)
- [RankByGene: Gene-Guided Histopathology Representation Learning Through Cross-Modal Ranking Consistency](https://mlsanity.com/paper/2411.15076/) (2024)
- [Completing Spatial Transcriptomics Data for Gene Expression Prediction Benchmarking](https://mlsanity.com/paper/2505.02980/) (2025)
- [GenAR: Next-Scale Autoregressive Generation for Spatial Gene Expression Prediction](https://mlsanity.com/paper/2510.04315/) (2025)
- [LAMMI-Pathology: A Tool-Centric Bottom-Up LVLM-Agent Framework for Molecularly Informed Medical Intelligence in Pathology](https://mlsanity.com/paper/2602.18773/) (2026)
- [SGI: Structured 2D Gaussians for Efficient and Compact Large Image Representation](https://mlsanity.com/paper/2603.07789/) (2026)
- [DELST: Dual Entailment Learning for Hyperbolic Image-Gene Pretraining in Spatial Transcriptomics](https://mlsanity.com/paper/2503.00804/) (2025)
- [MVHybrid: Improving Spatial Transcriptomics Prediction with Hybrid State Space-Vision Transformer Backbone in Pathology Vision Foundation Models](https://mlsanity.com/paper/2508.00383/) (2025)
- [HiFusion: Hierarchical Intra-Spot Alignment and Regional Context Fusion for Spatial Gene Expression Prediction from Histopathology](https://mlsanity.com/paper/2511.12969/) (2025)
- [HiST: A Hierarchical Sparse Transformer for Cross-Modal Spatial Transcriptomics Modeling](https://mlsanity.com/paper/2606.14251/) (2026)
- [SPATIA: Multimodal Generation and Prediction of Spatial Cell Phenotypes](https://mlsanity.com/paper/2507.04704/) (2025)
- [Learning from Gene Names, Expression Values and Images: Contrastive Masked Text-Image Pretraining for Spatial Transcriptomics Representation Learning](https://mlsanity.com/paper/2509.16892/) (2025)
- [Fusing Pixels and Genes: Spatially-Aware Learning in Computational Pathology](https://mlsanity.com/paper/2602.13944/) (2026)
- [Predicting Immune Biomarkers with MultiModal Mixture-of-Expert Pathology Foundation Models Empowers Precision Oncology](https://mlsanity.com/paper/2606.18123/) (2026)
- [Building artificial intelligence virtual tissue (AIVT) for tissue state representation, feature prediction, and dynamic simulation](https://mlsanity.com/paper/2606.29883/) (2026)
- [Correlation-Guided Flow Matching with Annealed Masking for Spatial Transcriptomics Generation](https://mlsanity.com/paper/2609.22187/) (2026)
- [MIST: Multi-Scale Spatial Context for Gene Expression Prediction from Histology](https://mlsanity.com/paper/2610.18769/) (2026)

## Abstract (arXiv)

Recent advances in multi-modal algorithms have driven and been driven by the increasing availability of large image-text datasets, leading to significant strides in various fields, including computational pathology. However, in most existing medical image-text datasets, the text typically provides high-level summaries that may not sufficiently describe sub-tile regions within a large pathology image. For example, an image might cover an extensive tissue area containing cancerous and healthy regions, but the accompanying text might only specify that this image is a cancer slide, lacking the nuanced details needed for in-depth analysis. In this study, we introduce STimage-1K4M, a novel dataset designed to bridge this gap by providing genomic features for sub-tile images. STimage-1K4M contains 1,149 images derived from spatial transcriptomics data, which captures gene expression information at the level of individual spatial spots within a pathology image. Specifically, each image in the dataset is broken down into smaller sub-image tiles, with each tile paired with 15,000-30,000 dimensional gene expressions. With 4,293,195 pairs of sub-tile images and gene expressions, STimage-1K4M offers unprecedented granularity, paving the way for a wide range of advanced research in multi-modal data analysis an innovative applications in computational pathology, and beyond.