# Semantic Segmentation for Histopathology using Learned Regularization based on Global Proportions

arXiv: https://arxiv.org/abs/2604.24347 · HTML: https://mlsanity.com/paper/2604.24347/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 32 of its 64 references that are in the map.
- First posted on arXiv 27 Apr 2026.

## Builds on (most cited first, 30 of 32)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization](https://mlsanity.com/paper/1606.00709/) (2016)
- [Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/1803.10464/) (2018)
- [Deep Unfolding Network for Image Super-Resolution](https://mlsanity.com/paper/2003.10428/) (2020)
- [Segment Anything in Medical Images](https://mlsanity.com/paper/2304.12306/) (2023)
- [Theoretical guarantees for approximate sampling from smooth and log-concave densities](https://mlsanity.com/paper/1412.7392/) (2014)
- [Better Aggregation in Test-Time Augmentation](https://mlsanity.com/paper/2011.11156/) (2020)
- [FISTA-Net: Learning A Fast Iterative Shrinkage Thresholding Network for Inverse Problems in Imaging](https://mlsanity.com/paper/2008.02683/) (2020)
- [Cluster-to-Conquer: A Framework for End-to-End Multi-Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2103.10626/) (2021)
- [Learning Loss for Test-Time Augmentation](https://mlsanity.com/paper/2010.11422/) (2020)
- [Total Deep Variation for Linear Inverse Problems](https://mlsanity.com/paper/2001.05005/) (2020)
- [Multi-Layer Pseudo-Supervision for Histopathology Tissue Semantic Segmentation using Patch-level Classification Labels](https://mlsanity.com/paper/2110.08048/) (2021)
- [A Comprehensive Analysis of Weakly-Supervised Semantic Segmentation in Different Image Domains](https://mlsanity.com/paper/1912.11186/) (2019)
- [Learning Weakly Convex Regularizers for Convergent Image-Reconstruction Algorithms](https://mlsanity.com/paper/2308.10542/) (2023)
- [Uncertainty Estimation via Response Scaling for Pseudo-mask Noise Mitigation in Weakly-supervised Semantic Segmentation](https://mlsanity.com/paper/2112.07431/) (2021)
- [Co-training for Demographic Classification Using Deep Learning from Label Proportions](https://mlsanity.com/paper/1709.04108/) (2017)
- [Equivariant Bootstrapping for Uncertainty Quantification in Imaging Inverse Problems](https://mlsanity.com/paper/2310.11838/) (2023)
- [Online Easy Example Mining for Weakly-supervised Gland Segmentation from Histology Images](https://mlsanity.com/paper/2206.06665/) (2022)
- [Weakly supervised segmentation with point annotations for histopathology images via contrast-based variational model](https://mlsanity.com/paper/2304.03572/) (2023)
- [All Points Matter: Entropy-Regularized Distribution Alignment for Weakly-supervised 3D Segmentation](https://mlsanity.com/paper/2305.15832/) (2023)
- [Negative Pseudo Labeling using Class Proportion for Semantic Segmentation in Pathology](https://mlsanity.com/paper/2007.08044/) (2020)
- [Learning from Label Proportion with Online Pseudo-Label Decision by Regret Minimization](https://mlsanity.com/paper/2302.08947/) (2023)
- [Learning from Partial Label Proportions for Whole Slide Image Segmentation](https://mlsanity.com/paper/2405.09041/) (2024)
- [Shared Prior Learning of Energy-Based Models for Image Reconstruction](https://mlsanity.com/paper/2011.06539/) (2020)
- [Generative Adversarial Networks for Stain Normalisation in Histopathology](https://mlsanity.com/paper/2308.02851/) (2023)

## Abstract (arXiv)

In pathology, the spatial distribution and proportions of tissue types are key indicators of disease progression, and are more readily available than fine-grained annotations. However, these assessments are rarely mapped to pixel-wise segmentation. The task is fundamentally underdetermined, as many spatially distinct segmentations can satisfy the same global proportions in the absence of pixel-wise constraints. To address this, we introduce Variational Segmentation from Label Proportions (VSLP), a two-stage framework that infers dense segmentations from global label proportions, without any pixel-level annotations. This framework first leverages a pre-trained transformer model with test-time augmentation to produce a pixel-wise confidence estimate. In the second stage, these estimates are fused by solving a variational optimization problem that incorporates a Wasserstein data fidelity term alongside a learned regularizer. Unlike end-to-end networks, our variational method can visualize the fidelity-regularization energy, resulting in more interpretable segmentation. We validate our approach on two public datasets, achieving superior performance over existing weakly supervised and unsupervised methods. For one of these datasets, proportions have been estimated by an experienced pathologist to provide a realistic benchmark to the community. Furthermore, the method scales to an in-house dataset with noisy pathologist labels, severely outperforming state-of-the-art methods, thereby demonstrating practical applicability. The code and data will be made publicly available upon acceptance at https://github.com/xiaoliangpi/VSLP.