# Unsupervised Total Variation Loss for Semi-supervised Deep Learning of Semantic Segmentation

arXiv: https://arxiv.org/abs/1605.01368 · HTML: https://mlsanity.com/paper/1605.01368/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 May 2016.

## Built on by (most cited first, 11 of 11)

- [PA-Seg: Learning from Point Annotations for 3D Medical Image Segmentation using Contextual Regularization and Cross Knowledge Distillation](https://mlsanity.com/paper/2208.05669/) (2022)
- [Ultrasound Nodule Segmentation Using Asymmetric Learning with Simple Clinical Annotation](https://mlsanity.com/paper/2404.14852/) (2024)
- [Scribble-based 3D Multiple Abdominal Organ Segmentation via Triple-branch Multi-dilated Network with Pixel- and Class-wise Consistency](https://mlsanity.com/paper/2309.09730/) (2023)
- [SAM-Driven Weakly Supervised Nodule Segmentation with Uncertainty-Aware Cross Teaching](https://mlsanity.com/paper/2407.13553/) (2024)
- [Pixel-Wise Color Constancy via Smoothness Techniques in Multi-Illuminant Scenes](https://mlsanity.com/paper/2402.02922/) (2024)
- [COIN: Counterfactual inpainting for weakly supervised semantic segmentation for medical images](https://mlsanity.com/paper/2404.12832/) (2024)
- [EAUWSeg: Eliminating annotation uncertainty in weakly-supervised medical image segmentation](https://mlsanity.com/paper/2501.01658/) (2025)
- [A Total Variation Regularized Framework for Epilepsy-Related MRI Image Segmentation](https://mlsanity.com/paper/2510.06276/) (2025)
- [CmFNet: Cross-modal Fusion Network for Weakly-supervised Segmentation of Medical Images](https://mlsanity.com/paper/2506.18042/) (2025)
- [Neural Artistic Style and Color Transfer Using Deep Learning](https://mlsanity.com/paper/2508.08608/) (2025)
- [Amortized Counterfactual Prediction with Graph Neural Networks](https://mlsanity.com/paper/2610.13724/) (2026)

## Abstract (arXiv)

We introduce a novel unsupervised loss function for learning semantic segmentation with deep convolutional neural nets (ConvNet) when densely labeled training images are not available. More specifically, the proposed loss function penalizes the L1-norm of the gradient of the label probability vector image , i.e. total variation, produced by the ConvNet. This can be seen as a regularization term that promotes piecewise smoothness of the label probability vector image produced by the ConvNet during learning. The unsupervised loss function is combined with a supervised loss in a semi-supervised setting to learn ConvNets that can achieve high semantic segmentation accuracy even when only a tiny percentage of the pixels in the training images are labeled. We demonstrate significant improvements over the purely supervised setting in the Weizmann horse, Stanford background and Sift Flow datasets. Furthermore, we show that using the proposed piecewise smoothness constraint in the learning phase significantly outperforms post-processing results from a purely supervised approach with Markov Random Fields (MRF). Finally, we note that the framework we introduce is general and can be used to learn to label other types of structures such as curvilinear structures by modifying the unsupervised loss function accordingly.