# Railroad is not a Train: Saliency as Pseudo-pixel Supervision for Weakly Supervised Semantic Segmentation

arXiv: https://arxiv.org/abs/2105.08965 · HTML: https://mlsanity.com/paper/2105.08965/

## Key facts

- Cited by 127 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 52 references that are in the map.
- First posted on arXiv 19 May 2021.

## Builds on (most cited first, 30 of 34)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [COCO-Stuff: Thing and Stuff Classes in Context](https://mlsanity.com/paper/1612.03716/) (2016)
- [Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs](https://mlsanity.com/paper/1412.7062/) (2014)
- [ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1604.05144/) (2016)
- [Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/1803.10464/) (2018)
- [Self-supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2004.04581/) (2020)
- [Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relations](https://mlsanity.com/paper/1904.05044/) (2019)
- [Simple Does It: Weakly Supervised Instance and Semantic Segmentation](https://mlsanity.com/paper/1603.07485/) (2016)
- [Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approach](https://mlsanity.com/paper/1703.08448/) (2017)
- [Seed, Expand and Constrain: Three Principles for Weakly-Supervised Image Segmentation](https://mlsanity.com/paper/1603.06098/) (2016)
- [Deeply supervised salient object detection with short connections](https://mlsanity.com/paper/1611.04849/) (2016)
- [Wider or Deeper: Revisiting the ResNet Model for Visual Recognition](https://mlsanity.com/paper/1611.10080/) (2016)
- [Tell Me Where to Look: Guided Attention Inference Network](https://mlsanity.com/paper/1802.10171/) (2018)
- [Single-Stage Semantic Segmentation from Image Labels](https://mlsanity.com/paper/2005.08104/) (2020)
- [FickleNet: Weakly and Semi-supervised Semantic Image Segmentation using Stochastic Inference](https://mlsanity.com/paper/1902.10421/) (2019)
- [Revisiting Dilated Convolution: A Simple Approach for Weakly- and Semi- Supervised Semantic Segmentation](https://mlsanity.com/paper/1805.04574/) (2018)
- [Weakly-Supervised Semantic Segmentation via Sub-category Exploration](https://mlsanity.com/paper/2008.01183/) (2020)
- [Mining Cross-Image Semantics for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2007.01947/) (2020)
- [Pyramid Feature Attention Network for Saliency detection](https://mlsanity.com/paper/1903.00179/) (2019)
- [Evaluating Weakly Supervised Object Localization Methods Right](https://mlsanity.com/paper/2001.07437/) (2020)
- [Constrained Convolutional Neural Networks for Weakly Supervised Segmentation](https://mlsanity.com/paper/1506.03648/) (2015)
- [Self-Erasing Network for Integral Object Attention](https://mlsanity.com/paper/1810.09821/) (2018)
- [Reliability Does Matter: An End-to-End Weakly Supervised Semantic Segmentation Approach](https://mlsanity.com/paper/1911.08039/) (2019)
- [Weakly-Supervised Semantic Segmentation by Iteratively Mining Common Object Features](https://mlsanity.com/paper/1806.04659/) (2018)
- [CIAN: Cross-Image Affinity Net for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/1811.10842/) (2018)
- [STC: A Simple to Complex Framework for Weakly-supervised Semantic Segmentation](https://mlsanity.com/paper/1509.03150/) (2015)
- [Joint Learning of Saliency Detection and Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/1909.04161/) (2019)
- [Exploiting saliency for object segmentation from image level labels](https://mlsanity.com/paper/1701.08261/) (2017)

## Built on by (most cited first, 30 of 127)

- [Multi-class Token Transformer for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2203.02891/) (2022)
- [Learning Affinity from Attention: End-to-End Weakly-Supervised Semantic Segmentation with Transformers](https://mlsanity.com/paper/2203.02664/) (2022)
- [CLIP is Also an Efficient Segmenter: A Text-Driven Approach for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2212.09506/) (2022)
- [Self-supervised Image-specific Prototype Exploration for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2203.02909/) (2022)
- [Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast](https://mlsanity.com/paper/2110.07110/) (2021)
- [Class Re-Activation Maps for Weakly-Supervised Semantic Segmentation](https://mlsanity.com/paper/2203.00962/) (2022)
- [Cross Language Image Matching for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2203.02668/) (2022)
- [Token Contrast for Weakly-Supervised Semantic Segmentation](https://mlsanity.com/paper/2303.01267/) (2023)
- [Regional Semantic Contrast and Aggregation for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2203.09653/) (2022)
- [LOCATE: Localize and Transfer Object Parts for Weakly Supervised Affordance Grounding](https://mlsanity.com/paper/2303.09665/) (2023)
- [Weakly Supervised Semantic Segmentation using Out-of-Distribution Data](https://mlsanity.com/paper/2203.03860/) (2022)
- [L2G: A Simple Local-to-Global Knowledge Transfer Framework for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2204.03206/) (2022)
- [Unsupervised Semantic Segmentation with Self-supervised Object-centric Representations](https://mlsanity.com/paper/2207.05027/) (2022)
- [Segment Anything Model (SAM) Enhanced Pseudo Labels for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2305.05803/) (2023)
- [ZeroWaste Dataset: Towards Deformable Object Segmentation in Cluttered Scenes](https://mlsanity.com/paper/2106.02740/) (2021)
- [Weakly-Supervised Camouflaged Object Detection with Scribble Annotations](https://mlsanity.com/paper/2207.14083/) (2022)
- [Max Pooling with Vision Transformers reconciles class and shape in weakly supervised semantic segmentation](https://mlsanity.com/paper/2210.17400/) (2022)
- [CLIP in Medical Imaging: A Survey](https://mlsanity.com/paper/2312.07353/) (2023)
- [Incremental Learning in Semantic Segmentation from Image Labels](https://mlsanity.com/paper/2112.01882/) (2021)
- [Threshold Matters in WSSS: Manipulating the Activation for the Robust and Accurate Segmentation Model Against Thresholds](https://mlsanity.com/paper/2203.16045/) (2022)
- [Expansion and Shrinkage of Localization for Weakly-Supervised Semantic Segmentation](https://mlsanity.com/paper/2209.07761/) (2022)
- [Going Beyond XAI: A Systematic Survey for Explanation-Guided Learning](https://mlsanity.com/paper/2212.03954/) (2022)
- [Foundation Model Assisted Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2312.03585/) (2023)
- [Extracting Class Activation Maps from Non-Discriminative Features as well](https://mlsanity.com/paper/2303.10334/) (2023)
- [Segment Anything is A Good Pseudo-label Generator for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2305.01275/) (2023)
- [Beyond Semantic to Instance Segmentation: Weakly-Supervised Instance Segmentation via Semantic Knowledge Transfer and Self-Refinement](https://mlsanity.com/paper/2109.09477/) (2021)
- [Out-of-Candidate Rectification for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2211.12268/) (2022)
- [Self Correspondence Distillation for End-to-End Weakly-Supervised Semantic Segmentation](https://mlsanity.com/paper/2302.13765/) (2023)
- [Semantic Segmentation In-the-Wild Without Seeing Any Segmentation Examples](https://mlsanity.com/paper/2112.03185/) (2021)
- [Texture-guided Saliency Distilling for Unsupervised Salient Object Detection](https://mlsanity.com/paper/2207.05921/) (2022)

## Abstract (arXiv)

Existing studies in weakly-supervised semantic segmentation (WSSS) using image-level weak supervision have several limitations: sparse object coverage, inaccurate object boundaries, and co-occurring pixels from non-target objects. To overcome these challenges, we propose a novel framework, namely Explicit Pseudo-pixel Supervision (EPS), which learns from pixel-level feedback by combining two weak supervisions; the image-level label provides the object identity via the localization map and the saliency map from the off-the-shelf saliency detection model offers rich boundaries. We devise a joint training strategy to fully utilize the complementary relationship between both information. Our method can obtain accurate object boundaries and discard co-occurring pixels, thereby significantly improving the quality of pseudo-masks. Experimental results show that the proposed method remarkably outperforms existing methods by resolving key challenges of WSSS and achieves the new state-of-the-art performance on both PASCAL VOC 2012 and MS COCO 2014 datasets.