# Modeling the Label Distributions for Weakly-Supervised Semantic Segmentation

arXiv: https://arxiv.org/abs/2403.13225 · HTML: https://mlsanity.com/paper/2403.13225/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Mar 2024.

## Built on by (most cited first, 7 of 7)

- [Large-Scale 3D Medical Image Pre-training with Geometric Context Priors](https://mlsanity.com/paper/2410.09890/) (2024)
- [FreeTumor: Advance Tumor Segmentation via Large-Scale Tumor Synthesis](https://mlsanity.com/paper/2406.01264/) (2024)
- [UniBiomed: A Universal Foundation Model for Grounded Biomedical Image Interpretation](https://mlsanity.com/paper/2504.21336/) (2025)
- [Soft Self-labeling and Potts Relaxations for Weakly-Supervised Segmentation](https://mlsanity.com/paper/2507.01721/) (2025)
- [UniVector: Unified Vector Extraction via Instance-Geometry Interaction](https://mlsanity.com/paper/2510.13234/) (2025)
- [Glance and Focus Reinforcement for Pan-cancer Screening](https://mlsanity.com/paper/2601.19103/) (2026)
- [TACoS: Weakly Supervised Learning of Two-Dimensional Materials from Scribble Annotations to Precise Segmentation](https://mlsanity.com/paper/2607.07169/) (2026)

## Abstract (arXiv)

Weakly-Supervised Semantic Segmentation (WSSS) aims to train segmentation models by weak labels, which is receiving significant attention due to its low annotation cost. Existing approaches focus on generating pseudo labels for supervision while largely ignoring to leverage the inherent semantic correlation among different pseudo labels. We observe that pseudo-labeled pixels that are close to each other in the feature space are more likely to share the same class, and those closer to the distribution centers tend to have higher confidence. Motivated by this, we propose to model the underlying label distributions and employ cross-label constraints to generate more accurate pseudo labels. In this paper, we develop a unified WSSS framework named Adaptive Gaussian Mixtures Model, which leverages a GMM to model the label distributions. Specifically, we calculate the feature distribution centers of pseudo-labeled pixels and build the GMM by measuring the distance between the centers and each pseudo-labeled pixel. Then, we introduce an Online Expectation-Maximization (OEM) algorithm and a novel maximization loss to optimize the GMM adaptively, aiming to learn more discriminative decision boundaries between different class-wise Gaussian mixtures. Based on the label distributions, we leverage the GMM to generate high-quality pseudo labels for more reliable supervision. Our framework is capable of solving different forms of weak labels: image-level labels, points, scribbles, blocks, and bounding-boxes. Extensive experiments on PASCAL, COCO, Cityscapes, and ADE20K datasets demonstrate that our framework can effectively provide more reliable supervision and outperform the state-of-the-art methods under all settings. Code will be available at https://github.com/Luffy03/AGMM-SASS.