# Automatic universal taxonomies for multi-domain semantic segmentation

arXiv: https://arxiv.org/abs/2207.08445 · HTML: https://mlsanity.com/paper/2207.08445/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 44 references that are in the map.
- First posted on arXiv 18 Jul 2022.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Language-driven Semantic Segmentation](https://mlsanity.com/paper/2201.03546/) (2022)
- [Green AI](https://mlsanity.com/paper/1907.10597/) (2019)
- [Rethinking ImageNet Pre-training](https://mlsanity.com/paper/1811.08883/) (2018)
- [Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Art](https://mlsanity.com/paper/1704.05519/) (2017)
- [IDD: A Dataset for Exploring Problems of Autonomous Navigation in Unconstrained Environments](https://mlsanity.com/paper/1811.10200/) (2018)
- [Entropy Maximization and Meta Classification for Out-Of-Distribution Detection in Semantic Segmentation](https://mlsanity.com/paper/2012.06575/) (2020)
- [Simple multi-dataset detection](https://mlsanity.com/paper/2102.13086/) (2021)
- [MSeg: A Composite Dataset for Multi-domain Semantic Segmentation](https://mlsanity.com/paper/2112.13762/) (2021)
- [Pixel-wise Anomaly Detection in Complex Driving Scenes](https://mlsanity.com/paper/2103.05445/) (2021)
- [Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes](https://mlsanity.com/paper/2101.06085/) (2021)
- [Pixel-wise Energy-biased Abstention Learning for Anomaly Segmentation on Complex Urban Driving Scenes](https://mlsanity.com/paper/2111.12264/) (2021)
- [In-Place Activated BatchNorm for Memory-Optimized Training of DNNs](https://mlsanity.com/paper/1712.02616/) (2017)
- [Object Detection with a Unified Label Space from Multiple Datasets](https://mlsanity.com/paper/2008.06614/) (2020)
- [Universal Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/1811.10323/) (2018)
- [LandCover.ai: Dataset for Automatic Mapping of Buildings, Woodlands, Water and Roads from Aerial Imagery](https://mlsanity.com/paper/2005.02264/) (2020)
- [Dynamic-structured Semantic Propagation Network](https://mlsanity.com/paper/1803.06067/) (2018)
- [Real-time Semantic Segmentation with Fast Attention](https://mlsanity.com/paper/2007.03815/) (2020)
- [Training of Convolutional Networks on Multiple Heterogeneous Datasets for Street Scene Semantic Segmentation](https://mlsanity.com/paper/1803.05675/) (2018)
- [Dense outlier detection and open-set recognition based on training with noisy negative images](https://mlsanity.com/paper/2101.09193/) (2021)
- [Multi-domain semantic segmentation with overlapping labels](https://mlsanity.com/paper/2108.11224/) (2021)
- [Learning Semantic Segmentation from Multiple Datasets with Label Shifts](https://mlsanity.com/paper/2202.14030/) (2022)
- [Towards Unsupervised Open World Semantic Segmentation](https://mlsanity.com/paper/2201.01073/) (2022)
- [Learning Fully Dense Neural Networks for Image Semantic Segmentation](https://mlsanity.com/paper/1905.08929/) (2019)
- [The Missing Link: Finding label relations across datasets](https://mlsanity.com/paper/2206.04453/) (2022)

## Built on by (most cited first, 7 of 7)

- [Universal Object Detection with Large Vision Model](https://mlsanity.com/paper/2212.09408/) (2022)
- [Weakly supervised training of universal visual concepts for multi-domain semantic segmentation](https://mlsanity.com/paper/2212.10340/) (2022)
- [Bridging Annotation Gaps: Transferring Labels to Align Object Detection Datasets](https://mlsanity.com/paper/2506.04737/) (2025)
- [An Empirical Study on Multi-Domain Robust Semantic Segmentation](https://mlsanity.com/paper/2212.04221/) (2022)
- [Automated Label Unification for Multi-Dataset Semantic Segmentation with GNNs](https://mlsanity.com/paper/2407.10534/) (2024)
- [The COTe score: A decomposable framework for evaluating Document Layout Analysis models](https://mlsanity.com/paper/2603.12718/) (2026)
- [Knowledge Distillation for Semantic Segmentation: A Label Space Unification Approach](https://mlsanity.com/paper/2502.19177/) (2025)

## Abstract (arXiv)

Training semantic segmentation models on multiple datasets has sparked a lot of recent interest in the computer vision community. This interest has been motivated by expensive annotations and a desire to achieve proficiency across multiple visual domains. However, established datasets have mutually incompatible labels which disrupt principled inference in the wild. We address this issue by automatic construction of universal taxonomies through iterative dataset integration. Our method detects subset-superset relationships between dataset-specific labels, and supports learning of sub-class logits by treating super-classes as partial labels. We present experiments on collections of standard datasets and demonstrate competitive generalization performance with respect to previous work.