# Prompt What You Need: Enhancing Segmentation in Rainy Scenes with Anchor-based Prompting

arXiv: https://arxiv.org/abs/2305.03902 · HTML: https://mlsanity.com/paper/2305.03902/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 28 references that are in the map.
- First posted on arXiv 6 May 2023.

## Builds on (most cited first, 17 of 17)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Training Region-based Object Detectors with Online Hard Example Mining](https://mlsanity.com/paper/1604.03540/) (2016)
- [Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision](https://mlsanity.com/paper/2106.01226/) (2021)
- [Semi-Supervised Semantic Segmentation with Cross-Consistency Training](https://mlsanity.com/paper/2003.09005/) (2020)
- [Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges](https://mlsanity.com/paper/1902.07830/) (2019)
- [Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels](https://mlsanity.com/paper/2203.03884/) (2022)
- [SegGPT: Segmenting Everything In Context](https://mlsanity.com/paper/2304.03284/) (2023)
- [ST++: Make Self-training Work Better for Semi-supervised Semantic Segmentation](https://mlsanity.com/paper/2106.05095/) (2021)
- [PseudoSeg: Designing Pseudo Labels for Semantic Segmentation](https://mlsanity.com/paper/2010.09713/) (2020)
- [Semi-Supervised Semantic Segmentation with High- and Low-level Consistency](https://mlsanity.com/paper/1908.05724/) (2019)
- [Semi-Supervised Semantic Segmentation with Pixel-Level Contrastive Learning from a Class-wise Memory Bank](https://mlsanity.com/paper/2104.13415/) (2021)
- [Pixel Contrastive-Consistent Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2108.09025/) (2021)
- [Re-distributing Biased Pseudo Labels for Semi-supervised Semantic Segmentation: A Baseline Investigation](https://mlsanity.com/paper/2107.11279/) (2021)
- [Mixture Domain Adaptation to Improve Semantic Segmentation in Real-World Surveillance](https://mlsanity.com/paper/2211.10119/) (2022)

## Built on by (most cited first, 1 of 1)

- [A Survey on Segment Anything Model (SAM): Vision Foundation Model Meets Prompt Engineering](https://mlsanity.com/paper/2306.06211/) (2023)

## Abstract (arXiv)

Semantic segmentation in rainy scenes is a challenging task due to the complex environment, class distribution imbalance, and limited annotated data. To address these challenges, we propose a novel framework that utilizes semi-supervised learning and pre-trained segmentation foundation model to achieve superior performance. Specifically, our framework leverages the semi-supervised model as the basis for generating raw semantic segmentation results, while also serving as a guiding force to prompt pre-trained foundation model to compensate for knowledge gaps with entropy-based anchors. In addition, to minimize the impact of irrelevant segmentation masks generated by the pre-trained foundation model, we also propose a mask filtering and fusion mechanism that optimizes raw semantic segmentation results based on the principle of minimum risk. The proposed framework achieves superior segmentation performance on the Rainy WCity dataset and is awarded the first prize in the sub-track of STRAIN in ICME 2023 Grand Challenges.