# Balanced Prompt Adaptation against Entropy-Induced Collapse for Test-Time Binary Segmentation

arXiv: https://arxiv.org/abs/2609.21743 · HTML: https://mlsanity.com/paper/2609.21743/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 38 references that are in the map.
- First posted on arXiv 18 Sep 2026.

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Open-Vocabulary Semantic Segmentation with Mask-adapted CLIP](https://mlsanity.com/paper/2210.04150/) (2022)
- [Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1710.05006/) (2017)
- [DenseCLIP: Language-Guided Dense Prediction with Context-Aware Prompting](https://mlsanity.com/paper/2112.01518/) (2021)
- [GroupViT: Semantic Segmentation Emerges from Text Supervision](https://mlsanity.com/paper/2202.11094/) (2022)
- [Efficient Test-Time Model Adaptation without Forgetting](https://mlsanity.com/paper/2204.02610/) (2022)
- [Confidence Regularized Self-Training](https://mlsanity.com/paper/1908.09822/) (2019)
- [Towards Stable Test-Time Adaptation in Dynamic Wild World](https://mlsanity.com/paper/2302.12400/) (2023)
- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)
- [Test-Time Adaptable Neural Networks for Robust Medical Image Segmentation](https://mlsanity.com/paper/2004.04668/) (2020)
- [On the test-time zero-shot generalization of vision-language models: Do we really need prompt learning?](https://mlsanity.com/paper/2405.02266/) (2024)
- [Boosting Vision-Language Models with Transduction](https://mlsanity.com/paper/2406.01837/) (2024)
- [WATT: Weight Average Test-Time Adaptation of CLIP](https://mlsanity.com/paper/2406.13875/) (2024)
- [Domain Adaptation for Semantic Segmentation via Class-Balanced Self-Training](https://mlsanity.com/paper/1810.07911/) (2018)
- [CLIPArTT: Adaptation of CLIP to New Domains at Test Time](https://mlsanity.com/paper/2405.00754/) (2024)
- [Histopath-C: Towards Realistic Domain Shifts for Histopathology Vision-Language Adaptation](https://mlsanity.com/paper/2601.12493/) (2026)
- [Test-Time Optimization for Domain Adaptive Open Vocabulary Segmentation](https://mlsanity.com/paper/2501.04696/) (2025)
- [What Drives Test-Time Adaptation for CLIP? A Controlled Empirical Study from an Update Perspective](https://mlsanity.com/paper/2606.14299/) (2026)
- [Dual-Modality Anchor-Guided Filtering for Test-time Prompt Tuning](https://mlsanity.com/paper/2604.12403/) (2026)
- [Robustifying Vision-Language Models via Test-Time Prompt Adaptation](https://mlsanity.com/paper/2607.09450/) (2026)

## Abstract (arXiv)

Entropy minimization is a standard objective for test-time adaptation (TTA), but it can fail in imbalanced binary segmentation. Unlike image classification, dense segmentation aggregates thousands of pixel predictions, allowing the larger predicted class to dominate the update, pull minority predictions toward itself, and produce a degenerate mask as predictions saturate and their entropy gradients vanish. We theoretically establish this collapse in a shared-shift model. This analysis motivates Balanced-Anchor Prompt Adaptation (BAPA), which combines two complementary modules. The Class-Balanced Anchors (CBA) module selects high-confidence anchors separately from each predicted class and gives foreground and background equal total loss weight, preventing the larger region from dominating the update. Dynamic Prompt Adaptation (DPA) refreshes these anchors after each prediction update and optimizes only text-side prompt residuals while keeping the vision-language encoders frozen. This prompt-only update refines the foreground-background decision boundary without altering the pretrained dense visual representation. Across experiments from four domains, BAPA achieves the highest mean Dice among the evaluated methods. Factorized ablations further validate the complementary roles of CBA and DPA, supporting balanced prompt adaptation as an effective alternative to entropy minimization for test-time binary segmentation.