# Cross-modal Diffusion Modelling for Super-resolved Spatial Transcriptomics

arXiv: https://arxiv.org/abs/2404.12973 · HTML: https://mlsanity.com/paper/2404.12973/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 10 of its 25 references that are in the map.
- First posted on arXiv 19 Apr 2024.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 10 of 10)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Models](https://mlsanity.com/paper/2104.14951/) (2021)
- [Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning](https://mlsanity.com/paper/2206.02647/) (2022)
- [Diffusion Models, Image Super-Resolution And Everything: A Survey](https://mlsanity.com/paper/2401.00736/) (2024)
- [DisC-Diff: Disentangled Conditional Diffusion Model for Multi-Contrast MRI Super-Resolution](https://mlsanity.com/paper/2303.13933/) (2023)
- [Multi-modal learning for predicting the genotype of glioma](https://mlsanity.com/paper/2203.10852/) (2022)
- [Multi-task Learning of Histology and Molecular Markers for Classifying Diffuse Glioma](https://mlsanity.com/paper/2303.14845/) (2023)

## Built on by (most cited first, 7 of 7)

- [Content Generation Models in Computational Pathology: A Comprehensive Survey on Methods, Applications, and Challenges](https://mlsanity.com/paper/2505.10993/) (2025)
- [Adaptive Spatial Transcriptomics Interpolation via Cross-modal Cross-slice Modeling](https://mlsanity.com/paper/2505.10729/) (2025)
- [Score-based Diffusion Model for Unpaired Virtual Histology Staining](https://mlsanity.com/paper/2506.23184/) (2025)
- [Gene-DML: Dual-Pathway Multi-Level Discrimination for Gene Expression Prediction from Histopathology Images](https://mlsanity.com/paper/2507.14670/) (2025)
- [Joint Modelling Histology and Molecular Markers for Cancer Classification](https://mlsanity.com/paper/2502.07979/) (2025)
- [HaDM-ST: Histology-Assisted Differential Modeling for Spatial Transcriptomics Generation](https://mlsanity.com/paper/2508.07225/) (2025)
- [SENCA-st: Integrating Spatial Transcriptomics and Histopathology with Cross Attention Shared Encoder for Region Identification in Cancer Pathology](https://mlsanity.com/paper/2511.08573/) (2025)

## Abstract (arXiv)

The recent advancement of spatial transcriptomics (ST) allows to characterize spatial gene expression within tissue for discovery research. However, current ST platforms suffer from low resolution, hindering in-depth understanding of spatial gene expression. Super-resolution approaches promise to enhance ST maps by integrating histology images with gene expressions of profiled tissue spots. However, current super-resolution methods are limited by restoration uncertainty and mode collapse. Although diffusion models have shown promise in capturing complex interactions between multi-modal conditions, it remains a challenge to integrate histology images and gene expression for super-resolved ST maps. This paper proposes a cross-modal conditional diffusion model for super-resolving ST maps with the guidance of histology images. Specifically, we design a multi-modal disentangling network with cross-modal adaptive modulation to utilize complementary information from histology images and spatial gene expression. Moreover, we propose a dynamic cross-attention modelling strategy to extract hierarchical cell-to-tissue information from histology images. Lastly, we propose a co-expression-based gene-correlation graph network to model the co-expression relationship of multiple genes. Experiments show that our method outperforms other state-of-the-art methods in ST super-resolution on three public datasets.