# Constrained-CNN losses for weakly supervised segmentation

arXiv: https://arxiv.org/abs/1805.04628 · HTML: https://mlsanity.com/paper/1805.04628/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 May 2018.

## Built on by (most cited first, 30 of 62)

- [Source-Free Domain Adaptation for Image Segmentation](https://mlsanity.com/paper/2108.03152/) (2021)
- [Robust Calibration of Large Vision-Language Adapters](https://mlsanity.com/paper/2407.13588/) (2024)
- [EAGLE: Eigen Aggregation Learning for Object-Centric Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2403.01482/) (2024)
- [Medical Image Segmentation with Limited Supervision: A Review of Deep Network Models](https://mlsanity.com/paper/2103.00429/) (2021)
- [Inter Extreme Points Geodesics for End-to-End Weakly Supervised Image Segmentation](https://mlsanity.com/paper/2107.00583/) (2021)
- [Constrained unsupervised anomaly segmentation](https://mlsanity.com/paper/2203.01671/) (2022)
- [Test-Time Adaptation with Shape Moments for Image Segmentation](https://mlsanity.com/paper/2205.07983/) (2022)
- [Local Spatiotemporal Representation Learning for Longitudinally-consistent Neuroimage Analysis](https://mlsanity.com/paper/2206.04281/) (2022)
- [Weakly supervised segmentation with cross-modality equivariant constraints](https://mlsanity.com/paper/2104.02488/) (2021)
- [PA-Seg: Learning from Point Annotations for 3D Medical Image Segmentation using Contextual Regularization and Cross Knowledge Distillation](https://mlsanity.com/paper/2208.05669/) (2022)
- [Prototype-Based Image Prompting for Weakly Supervised Histopathological Image Segmentation](https://mlsanity.com/paper/2503.12068/) (2025)
- [Scribble2D5: Weakly-Supervised Volumetric Image Segmentation via Scribble Annotations](https://mlsanity.com/paper/2205.06779/) (2022)
- [Weakly supervised segmentation with point annotations for histopathology images via contrast-based variational model](https://mlsanity.com/paper/2304.03572/) (2023)
- [Non-Iterative Scribble-Supervised Learning with Pacing Pseudo-Masks for Medical Image Segmentation](https://mlsanity.com/paper/2210.10956/) (2022)
- [Anatomy-guided domain adaptation for 3D in-bed human pose estimation](https://mlsanity.com/paper/2211.12193/) (2022)
- [Weakly-Supervised 3D Medical Image Segmentation using Geometric Prior and Contrastive Similarity](https://mlsanity.com/paper/2302.02125/) (2023)
- [Negative Evidence Matters in Interpretable Histology Image Classification](https://mlsanity.com/paper/2201.02445/) (2022)
- [Recent Advances in Fibrosis and Scar Segmentation from Cardiac MRI: A State-of-the-Art Review and Future Perspectives](https://mlsanity.com/paper/2106.15707/) (2021)
- [Re-using Adversarial Mask Discriminators for Test-time Training under Distribution Shifts](https://mlsanity.com/paper/2108.11926/) (2021)
- [Segmentation with mixed supervision: Confidence maximization helps knowledge distillation](https://mlsanity.com/paper/2109.10902/) (2021)
- [Beyond pixel-wise supervision for segmentation: A few global shape descriptors might be surprisingly good!](https://mlsanity.com/paper/2105.00859/) (2021)
- [Self-supervised Multi-scale Consistency for Weakly Supervised Segmentation Learning](https://mlsanity.com/paper/2108.11900/) (2021)
- [Stop Throwing Away Discriminators! Re-using Adversaries for Test-Time Training](https://mlsanity.com/paper/2108.12280/) (2021)
- [Looking at the whole picture: constrained unsupervised anomaly segmentation](https://mlsanity.com/paper/2109.00482/) (2021)
- [Annotation by Clicks: A Point-Supervised Contrastive Variance Method for Medical Semantic Segmentation](https://mlsanity.com/paper/2212.08774/) (2022)
- [Tackling Ambiguity from Perspective of Uncertainty Inference and Affinity Diversification for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2404.08195/) (2024)
- [Beyond Pixel-Wise Supervision for Medical Image Segmentation: From Traditional Models to Foundation Models](https://mlsanity.com/paper/2404.13239/) (2024)
- [Prompting classes: Exploring the Power of Prompt Class Learning in Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2307.00097/) (2023)
- [Boundary-RL: Reinforcement Learning for Weakly-Supervised Prostate Segmentation in TRUS Images](https://mlsanity.com/paper/2308.11376/) (2023)
- [Leveraging Anatomical Constraints with Uncertainty for Pneumothorax Segmentation](https://mlsanity.com/paper/2311.15213/) (2023)

## Abstract (arXiv)

Weakly-supervised learning based on, e.g., partially labelled images or image-tags, is currently attracting significant attention in CNN segmentation as it can mitigate the need for full and laborious pixel/voxel annotations. Enforcing high-order (global) inequality constraints on the network output (for instance, to constrain the size of the target region) can leverage unlabeled data, guiding the training process with domain-specific knowledge. Inequality constraints are very flexible because they do not assume exact prior knowledge. However, constrained Lagrangian dual optimization has been largely avoided in deep networks, mainly for computational tractability reasons. To the best of our knowledge, the method of [Pathak et al., 2015] is the only prior work that addresses deep CNNs with linear constraints in weakly supervised segmentation. It uses the constraints to synthesize fully-labeled training masks (proposals) from weak labels, mimicking full supervision and facilitating dual optimization. We propose to introduce a differentiable penalty, which enforces inequality constraints directly in the loss function, avoiding expensive Lagrangian dual iterates and proposal generation. From constrained-optimization perspective, our simple penalty-based approach is not optimal as there is no guarantee that the constraints are satisfied. However, surprisingly, it yields substantially better results than the Lagrangian-based constrained CNNs in [Pathak et al., 2015], while reducing the computational demand for training. By annotating only a small fraction of the pixels, the proposed approach can reach a level of segmentation performance that is comparable to full supervision on three separate tasks. While our experiments focused on basic linear constraints such as the target-region size and image tags, our framework can be easily extended to other non-linear constraints.