# Joint Learning of Saliency Detection and Weakly Supervised Semantic Segmentation

arXiv: https://arxiv.org/abs/1909.04161 · HTML: https://mlsanity.com/paper/1909.04161/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Sep 2019.

## Built on by (most cited first, 30 of 57)

- [Salient Object Detection via Integrity Learning](https://mlsanity.com/paper/2101.07663/) (2021)
- [Railroad is not a Train: Saliency as Pseudo-pixel Supervision for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2105.08965/) (2021)
- [Uncertainty-aware Joint Salient Object and Camouflaged Object Detection](https://mlsanity.com/paper/2104.02628/) (2021)
- [Large-scale Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2106.03149/) (2021)
- [Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast](https://mlsanity.com/paper/2110.07110/) (2021)
- [Leveraging Auxiliary Tasks with Affinity Learning for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2107.11787/) (2021)
- [Regional Semantic Contrast and Aggregation for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2203.09653/) (2022)
- [Pseudo-mask Matters in Weakly-supervised Semantic Segmentation](https://mlsanity.com/paper/2108.12995/) (2021)
- [Advances in Deep Concealed Scene Understanding](https://mlsanity.com/paper/2304.11234/) (2023)
- [Rethinking Semi-Supervised Medical Image Segmentation: A Variance-Reduction Perspective](https://mlsanity.com/paper/2302.01735/) (2023)
- [Background-Aware Pooling and Noise-Aware Loss for Weakly-Supervised Semantic Segmentation](https://mlsanity.com/paper/2104.00905/) (2021)
- [Threshold Matters in WSSS: Manipulating the Activation for the Robust and Accurate Segmentation Model Against Thresholds](https://mlsanity.com/paper/2203.16045/) (2022)
- [Class-Aware Adversarial Transformers for Medical Image Segmentation](https://mlsanity.com/paper/2201.10737/) (2022)
- [Group Collaborative Learning for Co-Salient Object Detection](https://mlsanity.com/paper/2104.01108/) (2021)
- [Mine yOur owN Anatomy: Revisiting Medical Image Segmentation with Extremely Limited Labels](https://mlsanity.com/paper/2209.13476/) (2022)
- [Unsupervised Foreground Extraction via Deep Region Competition](https://mlsanity.com/paper/2110.15497/) (2021)
- [Adaptive Affinity Loss and Erroneous Pseudo-Label Refinement for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2108.01344/) (2021)
- [Weakly-supervised segmentation of referring expressions](https://mlsanity.com/paper/2205.04725/) (2022)
- [Weakly-supervised Instance Segmentation via Class-agnostic Learning with Salient Images](https://mlsanity.com/paper/2104.01526/) (2021)
- [Salient Objects in Clutter](https://mlsanity.com/paper/2105.03053/) (2021)
- [Saliency Guided Inter- and Intra-Class Relation Constraints for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2206.09554/) (2022)
- [Weakly-Supervised Semantic Segmentation with Image-Level Labels: from Traditional Models to Foundation Models](https://mlsanity.com/paper/2310.13026/) (2023)
- [Sketch2Saliency: Learning to Detect Salient Objects from Human Drawings](https://mlsanity.com/paper/2303.11502/) (2023)
- [BiconNet: An Edge-preserved Connectivity-based Approach for Salient Object Detection](https://mlsanity.com/paper/2103.00334/) (2021)
- [Online Easy Example Mining for Weakly-supervised Gland Segmentation from Histology Images](https://mlsanity.com/paper/2206.06665/) (2022)
- [Panoramic Vision Transformer for Saliency Detection in 360deg Videos](https://mlsanity.com/paper/2209.08956/) (2022)
- [Cross-Image Region Mining with Region Prototypical Network for Weakly Supervised Segmentation](https://mlsanity.com/paper/2108.07413/) (2021)
- [GCoNet+: A Stronger Group Collaborative Co-Salient Object Detector](https://mlsanity.com/paper/2205.15469/) (2022)
- [Memory-aided Contrastive Consensus Learning for Co-salient Object Detection](https://mlsanity.com/paper/2302.14485/) (2023)
- [Co-Salient Object Detection with Co-Representation Purification](https://mlsanity.com/paper/2303.07670/) (2023)

## Abstract (arXiv)

Existing weakly supervised semantic segmentation (WSSS) methods usually utilize the results of pre-trained saliency detection (SD) models without explicitly modeling the connections between the two tasks, which is not the most efficient configuration. Here we propose a unified multi-task learning framework to jointly solve WSSS and SD using a single network, ie saliency, and segmentation network (SSNet). SSNet consists of a segmentation network (SN) and a saliency aggregation module (SAM). For an input image, SN generates the segmentation result and, SAM predicts the saliency of each category and aggregating the segmentation masks of all categories into a saliency map. The proposed network is trained end-to-end with image-level category labels and class-agnostic pixel-level saliency labels. Experiments on PASCAL VOC 2012 segmentation dataset and four saliency benchmark datasets show the performance of our method compares favorably against state-of-the-art weakly supervised segmentation methods and fully supervised saliency detection methods.