# HoloHisto: End-to-end Gigapixel WSI Segmentation with 4K Resolution Sequential Tokenization arXiv: https://arxiv.org/abs/2407.03307 · HTML: https://mlsanity.com/paper/2407.03307/ ## Key facts - Cited by 8 papers in the map (data through 14 Oct 2026). - Builds on 23 of its 35 references that are in the map. - First posted on arXiv 3 Jul 2024. ## Builds on (most cited first, 23 of 23) - [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020) - [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015) - [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023) - [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023) - [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021) - [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023) - [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017) - [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020) - [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018) - [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021) - [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020) - [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021) - [Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis](https://mlsanity.com/paper/2111.14791/) (2021) - [Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning](https://mlsanity.com/paper/2206.02647/) (2022) - [Sequential Modeling Enables Scalable Learning for Large Vision Models](https://mlsanity.com/paper/2312.00785/) (2023) - [Review of Large Vision Models and Visual Prompt Engineering](https://mlsanity.com/paper/2307.00855/) (2023) - [Self-Supervised Vision Transformers Learn Visual Concepts in Histopathology](https://mlsanity.com/paper/2203.00585/) (2022) - [The Multi-modality Cell Segmentation Challenge: Towards Universal Solutions](https://mlsanity.com/paper/2308.05864/) (2023) - [High Quality Segmentation for Ultra High-resolution Images](https://mlsanity.com/paper/2111.14482/) (2021) - [Omni-Seg: A Scale-aware Dynamic Network for Renal Pathological Image Segmentation](https://mlsanity.com/paper/2206.13632/) (2022) - [Data-efficient Large Vision Models through Sequential Autoregression](https://mlsanity.com/paper/2402.04841/) (2024) - [PrPSeg: Universal Proposition Learning for Panoramic Renal Pathology Segmentation](https://mlsanity.com/paper/2402.19286/) (2024) - [CNN Cascades for Segmenting Whole Slide Images of the Kidney](https://mlsanity.com/paper/1708.00251/) (2017) ## Built on by (most cited first, 8 of 8) - [Continual Alignment for SAM: Rethinking Foundation Models for Medical Image Segmentation in Continual Learning](https://mlsanity.com/paper/2511.17201/) (2025) - [Efficient Deep Learning for Medical Imaging: Bridging the Gap Between High-Performance AI and Clinical Deployment](https://mlsanity.com/paper/2602.00910/) (2026) - [MuViT: Multi-Resolution Vision Transformers for Learning Across Scales in Microscopy](https://mlsanity.com/paper/2602.24222/) (2026) - [Improved Baselines with Synchronized Encoding for Universal Medical Image Segmentation](https://mlsanity.com/paper/2408.09886/) (2024) - [A General Pipeline for Glomerulus Whole-Slide Image Segmentation](https://mlsanity.com/paper/2411.04782/) (2024) - [CTI-Unet: Cascaded Threshold Integration for Improved U-Net Segmentation of Pathology Images](https://mlsanity.com/paper/2504.05640/) (2025) - [Clinical-Injection Transformer with Domain-Adapted MAE for Lupus Nephritis Prognosis Prediction](https://mlsanity.com/paper/2603.05535/) (2026) - [CoordFormer: Give Me Any Coordinates and I Will Give You Labels](https://mlsanity.com/paper/2609.08660/) (2026) ## Abstract (arXiv) In digital pathology, the traditional method for deep learning-based image segmentation typically involves a two-stage process: initially segmenting high-resolution whole slide images (WSI) into smaller patches (e.g., 256x256, 512x512, 1024x1024) and subsequently reconstructing them to their original scale. This method often struggles to capture the complex details and vast scope of WSIs. In this paper, we propose the holistic histopathology (HoloHisto) segmentation method to achieve end-to-end segmentation on gigapixel WSIs, whose maximum resolution is above 80,000×70,000 pixels. HoloHisto fundamentally shifts the paradigm of WSI segmentation to an end-to-end learning fashion with 1) a large (4K) resolution base patch for elevated visual information inclusion and efficient processing, and 2) a novel sequential tokenization mechanism to properly model the contextual relationships and efficiently model the rich information from the 4K input. To our best knowledge, HoloHisto presents the first holistic approach for gigapixel resolution WSI segmentation, supporting direct I/O of complete WSI and their corresponding gigapixel masks. Under the HoloHisto platform, we unveil a random 4K sampler that transcends ultra-high resolution, delivering 31 and 10 times more pixels than standard 2D and 3D patches, respectively, for advancing computational capabilities. To facilitate efficient 4K resolution dense prediction, we leverage sequential tokenization, utilizing a pre-trained image tokenizer to group image features into a discrete token grid. To assess the performance, our team curated a new kidney pathology image segmentation (KPIs) dataset with WSI-level glomeruli segmentation from whole mouse kidneys. From the results, HoloHisto-4K delivers remarkable performance gains over previous state-of-the-art models.