# HiST: A Hierarchical Sparse Transformer for Cross-Modal Spatial Transcriptomics Modeling

arXiv: https://arxiv.org/abs/2606.14251 · HTML: https://mlsanity.com/paper/2606.14251/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 58 references that are in the map.
- First posted on arXiv 12 Jun 2026.

## Builds on (most cited first, 17 of 17)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation](https://mlsanity.com/paper/2108.12409/) (2021)
- [ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders](https://mlsanity.com/paper/2301.00808/) (2023)
- [Understanding the Effective Receptive Field in Deep Convolutional Neural Networks](https://mlsanity.com/paper/1701.04128/) (2017)
- [MambaVision: A Hybrid Mamba-Transformer Vision Backbone](https://mlsanity.com/paper/2407.08083/) (2024)
- [HEST-1k: A Dataset for Spatial Transcriptomics and Histology Image Analysis](https://mlsanity.com/paper/2406.16192/) (2024)
- [Spatially Resolved Gene Expression Prediction from H&E Histology Images via Bi-modal Contrastive Learning](https://mlsanity.com/paper/2306.01859/) (2023)
- [Diffusion Generative Modeling for Spatially Resolved Gene Expression Inference from Histology Images](https://mlsanity.com/paper/2501.15598/) (2025)
- [STimage-1K4M: A histopathology image-gene expression dataset for spatial transcriptomics](https://mlsanity.com/paper/2406.06393/) (2024)
- [Scalable Generation of Spatial Transcriptomics from Histology Images via Whole-Slide Flow Matching](https://mlsanity.com/paper/2506.05361/) (2025)
- [Improving Representation Learning for Histopathologic Images with Cluster Constraints](https://mlsanity.com/paper/2310.12334/) (2023)
- [Learning Spatial-Preserving Hierarchical Representations for Digital Pathology](https://mlsanity.com/paper/2406.09333/) (2024)
- [HECLIP: Histology-Enhanced Contrastive Learning for Imputation of Transcriptomics Profiles](https://mlsanity.com/paper/2501.14948/) (2025)
- [Exploiting Label-Independent Regularization from Spatial Dependencies for Whole Slide Image Analysis](https://mlsanity.com/paper/2602.19487/) (2026)

## Built on by (most cited first, 1 of 1)

- [Is H&E Image-to-Spatial Transcriptomics Simpler Than It Looks?](https://mlsanity.com/paper/2609.32857/) (2026)

## Abstract (arXiv)

Spatial transcriptomics (ST) links gene expression with tissue morphology but remains expensive and low-throughput, motivating surrogates that infer expression from routine histology. Whole-slide H&E-to-ST inference pairs a gigapixel image with gene measurements at a sparse, irregular set of locations, making multiscale modeling challenging without incurring dense-grid overhead or quadratic token mixing. We propose HiST, a hierarchical sparse transformer that treats measured locations as a lattice-indexed sparse field and builds a dyadic encoder--decoder directly on the active tissue footprint. HiST combines sparse window attention for local geometric correspondence with resolution-changing operators for rapid multiscale context integration. For a fixed window size, the dominant runtime and memory scale with the number of observed locations rather than the dense slide area. To mitigate slide-specific acquisition variation, HiST adds a bottlenecked global conditioning pathway via a slide calibration token that summarizes slide-level context and conditions local representations. On a multi-organ benchmark spanning diverse tissues and acquisition sources, HiST improves predictive performance over recent baselines while reducing runtime and peak memory.