# Tree Energy Loss: Towards Sparsely Annotated Semantic Segmentation

arXiv: https://arxiv.org/abs/2203.10739 · HTML: https://mlsanity.com/paper/2203.10739/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 46 references that are in the map.
- First posted on arXiv 21 Mar 2022.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [Holistically-Nested Edge Detection](https://mlsanity.com/paper/1504.06375/) (2015)
- [Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results](https://mlsanity.com/paper/1703.01780/) (2017)
- [ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1604.05144/) (2016)
- [What's the Point: Semantic Segmentation with Point Supervision](https://mlsanity.com/paper/1506.02106/) (2015)
- [Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation](https://mlsanity.com/paper/1901.02985/) (2019)
- [Exploring Cross-Image Pixel Contrast for Semantic Segmentation](https://mlsanity.com/paper/2101.11939/) (2021)
- [High-Resolution Representations for Labeling Pixels and Regions](https://mlsanity.com/paper/1904.04514/) (2019)
- [BoxInst: High-Performance Instance Segmentation with Box Annotations](https://mlsanity.com/paper/2012.02310/) (2020)
- [On Regularized Losses for Weakly-supervised CNN Segmentation](https://mlsanity.com/paper/1803.09569/) (2018)
- [Normalized Cut Loss for Weakly-supervised CNN Segmentation](https://mlsanity.com/paper/1804.01346/) (2018)
- [Learning random-walk label propagation for weakly-supervised semantic segmentation](https://mlsanity.com/paper/1802.00470/) (2018)
- [Regional Semantic Contrast and Aggregation for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2203.09653/) (2022)
- [Affinity Attention Graph Neural Network for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2106.04054/) (2021)
- [Universal Weakly Supervised Segmentation by Pixel-to-Segment Contrastive Learning](https://mlsanity.com/paper/2105.00957/) (2021)
- [Gated CRF Loss for Weakly Supervised Semantic Image Segmentation](https://mlsanity.com/paper/1906.04651/) (2019)
- [Weakly Supervised Scene Parsing with Point-based Distance Metric Learning](https://mlsanity.com/paper/1811.02233/) (2018)
- [Learnable Tree Filter for Structure-preserving Feature Transform](https://mlsanity.com/paper/1909.12513/) (2019)
- [Scribble-Supervised Semantic Segmentation by Uncertainty Reduction on Neural Representation and Self-Supervision on Neural Eigenspace](https://mlsanity.com/paper/2102.09896/) (2021)
- [Seminar Learning for Click-Level Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2108.13393/) (2021)
- [Rethinking Learnable Tree Filter for Generic Feature Transform](https://mlsanity.com/paper/2012.03482/) (2020)
- [WeClick: Weakly-Supervised Video Semantic Segmentation with Click Annotations](https://mlsanity.com/paper/2107.03088/) (2021)

## Built on by (most cited first, 30 of 47)

- [Weakly-Supervised Concealed Object Segmentation with SAM-based Pseudo Labeling and Multi-scale Feature Grouping](https://mlsanity.com/paper/2305.11003/) (2023)
- [Box-supervised Instance Segmentation with Level Set Evolution](https://mlsanity.com/paper/2207.09055/) (2022)
- [Point2Mask: Point-supervised Panoptic Segmentation via Optimal Transport](https://mlsanity.com/paper/2308.01779/) (2023)
- [Box2Mask: Box-supervised Instance Segmentation via Level-set Evolution](https://mlsanity.com/paper/2212.01579/) (2022)
- [SAM-COD: SAM-guided Unified Framework for Weakly-Supervised Camouflaged Object Detection](https://mlsanity.com/paper/2408.10760/) (2024)
- [LiDAR2Map: In Defense of LiDAR-Based Semantic Map Construction Using Online Camera Distillation](https://mlsanity.com/paper/2304.11379/) (2023)
- [Frozen CLIP: A Strong Backbone for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2406.11189/) (2024)
- [Pixel Difference Convolutional Network for RGB-D Semantic Segmentation](https://mlsanity.com/paper/2302.11951/) (2023)
- [Learning Camouflaged Object Detection from Noisy Pseudo Label](https://mlsanity.com/paper/2407.13157/) (2024)
- [Mutual Information Regularization for Weakly-supervised RGB-D Salient Object Detection](https://mlsanity.com/paper/2306.03630/) (2023)
- [Label-efficient Semantic Scene Completion with Scribble Annotations](https://mlsanity.com/paper/2405.15170/) (2024)
- [Scribble-based 3D Multiple Abdominal Organ Segmentation via Triple-branch Multi-dilated Network with Pixel- and Class-wise Consistency](https://mlsanity.com/paper/2309.09730/) (2023)
- [Label-efficient Segmentation via Affinity Propagation](https://mlsanity.com/paper/2310.10533/) (2023)
- [SASFormer: Transformers for Sparsely Annotated Semantic Segmentation](https://mlsanity.com/paper/2212.02019/) (2022)
- [FedLPPA: Learning Personalized Prompt and Aggregation for Federated Weakly-supervised Medical Image Segmentation](https://mlsanity.com/paper/2402.17502/) (2024)
- [Scribble Hides Class: Promoting Scribble-Based Weakly-Supervised Semantic Segmentation with Its Class Label](https://mlsanity.com/paper/2402.17555/) (2024)
- [SCALER: SAM-Enhanced Collaborative Learning for Label-Deficient Concealed Object Segmentation](https://mlsanity.com/paper/2511.18136/) (2025)
- [ScribbleGen: Generative Data Augmentation Improves Scribble-supervised Semantic Segmentation](https://mlsanity.com/paper/2311.17121/) (2023)
- [Unifying and Personalizing Weakly-supervised Federated Medical Image Segmentation via Adaptive Representation and Aggregation](https://mlsanity.com/paper/2304.05635/) (2023)
- [Scribbles for All: Benchmarking Scribble Supervised Segmentation Across Datasets](https://mlsanity.com/paper/2408.12489/) (2024)
- [Segment Concealed Objects with Incomplete Supervision](https://mlsanity.com/paper/2506.08955/) (2025)
- [A Language-Guided Benchmark for Weakly Supervised Open Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2302.14163/) (2023)
- [ProCNS: Progressive Prototype Calibration and Noise Suppression for Weakly-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2401.14074/) (2024)
- [Towards Real Zero-Shot Camouflaged Object Segmentation without Camouflaged Annotations](https://mlsanity.com/paper/2410.16953/) (2024)
- [Stepwise Decomposition and Dual-stream Focus: A Novel Approach for Training-free Camouflaged Object Segmentation](https://mlsanity.com/paper/2506.06818/) (2025)
- [Segment Anything Across Shots: A Method and Benchmark](https://mlsanity.com/paper/2511.13715/) (2025)
- [Boosting Weakly-Supervised Image Segmentation via Representation, Transform, and Compensator](https://mlsanity.com/paper/2309.00871/) (2023)
- [Soft Self-labeling and Potts Relaxations for Weakly-Supervised Segmentation](https://mlsanity.com/paper/2507.01721/) (2025)
- [RecurSeed and EdgePredictMix: Pseudo-Label Refinement Learning for Weakly Supervised Semantic Segmentation across Single- and Multi-Stage Frameworks](https://mlsanity.com/paper/2204.06754/) (2022)
- [Land Cover Segmentation with Sparse Annotations from Sentinel-2 Imagery](https://mlsanity.com/paper/2306.16252/) (2023)

## Abstract (arXiv)

Sparsely annotated semantic segmentation (SASS) aims to train a segmentation network with coarse-grained (i.e., point-, scribble-, and block-wise) supervisions, where only a small proportion of pixels are labeled in each image. In this paper, we propose a novel tree energy loss for SASS by providing semantic guidance for unlabeled pixels. The tree energy loss represents images as minimum spanning trees to model both low-level and high-level pair-wise affinities. By sequentially applying these affinities to the network prediction, soft pseudo labels for unlabeled pixels are generated in a coarse-to-fine manner, achieving dynamic online self-training. The tree energy loss is effective and easy to be incorporated into existing frameworks by combining it with a traditional segmentation loss. Compared with previous SASS methods, our method requires no multistage training strategies, alternating optimization procedures, additional supervised data, or time-consuming post-processing while outperforming them in all SASS settings. Code is available at https://github.com/megvii-research/TreeEnergyLoss.