# Ada-Segment: Automated Multi-loss Adaptation for Panoptic Segmentation

arXiv: https://arxiv.org/abs/2012.03603 · HTML: https://mlsanity.com/paper/2012.03603/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Dec 2020.

## Built on by (most cited first, 3 of 3)

- [Panoptic Segmentation: A Review](https://mlsanity.com/paper/2111.10250/) (2021)
- [HLIC: Harmonizing Optimization Metrics in Learned Image Compression by Reinforcement Learning](https://mlsanity.com/paper/2109.14863/) (2021)
- [CFNet: Learning Correlation Functions for One-Stage Panoptic Segmentation](https://mlsanity.com/paper/2201.04796/) (2022)

## Abstract (arXiv)

Panoptic segmentation that unifies instance segmentation and semantic segmentation has recently attracted increasing attention. While most existing methods focus on designing novel architectures, we steer toward a different perspective: performing automated multi-loss adaptation (named Ada-Segment) on the fly to flexibly adjust multiple training losses over the course of training using a controller trained to capture the learning dynamics. This offers a few advantages: it bypasses manual tuning of the sensitive loss combination, a decisive factor for panoptic segmentation; it allows to explicitly model the learning dynamics, and reconcile the learning of multiple objectives (up to ten in our experiments); with an end-to-end architecture, it generalizes to different datasets without the need of re-tuning hyperparameters or re-adjusting the training process laboriously. Our Ada-Segment brings 2.7% panoptic quality (PQ) improvement on COCO val split from the vanilla baseline, achieving the state-of-the-art 48.5% PQ on COCO test-dev split and 32.9% PQ on ADE20K dataset. The extensive ablation studies reveal the ever-changing dynamics throughout the training process, necessitating the incorporation of an automated and adaptive learning strategy as presented in this paper.