# Bridging Annotation Gaps: Transferring Labels to Align Object Detection Datasets

arXiv: https://arxiv.org/abs/2506.04737 · HTML: https://mlsanity.com/paper/2506.04737/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 35 references that are in the map.
- First posted on arXiv 5 Jun 2025.

## Builds on (most cited first, 30 of 31)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Do ImageNet Classifiers Generalize to ImageNet?](https://mlsanity.com/paper/1902.10811/) (2019)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [Are we done with ImageNet?](https://mlsanity.com/paper/2006.07159/) (2020)
- [Unbiased Teacher for Semi-Supervised Object Detection](https://mlsanity.com/paper/2102.09480/) (2021)
- [Mind the Class Weight Bias: Weighted Maximum Mean Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1705.00609/) (2017)
- [SHIFT: A Synthetic Driving Dataset for Continuous Multi-Task Domain Adaptation](https://mlsanity.com/paper/2206.08367/) (2022)
- [Simple multi-dataset detection](https://mlsanity.com/paper/2102.13086/) (2021)
- [Cross-Domain Adaptive Teacher for Object Detection](https://mlsanity.com/paper/2111.13216/) (2021)
- [MSeg: A Composite Dataset for Multi-domain Semantic Segmentation](https://mlsanity.com/paper/2112.13762/) (2021)
- [Re-labeling ImageNet: from Single to Multi-Labels, from Global to Localized Labels](https://mlsanity.com/paper/2101.05022/) (2021)
- [Synscapes: A Photorealistic Synthetic Dataset for Street Scene Parsing](https://mlsanity.com/paper/1810.08705/) (2018)
- [Towards Universal Object Detection by Domain Attention](https://mlsanity.com/paper/1904.04402/) (2019)
- [PASTA: Proportional Amplitude Spectrum Training Augmentation for Syn-to-Real Domain Generalization](https://mlsanity.com/paper/2212.00979/) (2022)
- [2PCNet: Two-Phase Consistency Training for Day-to-Night Unsupervised Domain Adaptive Object Detection](https://mlsanity.com/paper/2303.13853/) (2023)
- [Detection Hub: Unifying Object Detection Datasets via Query Adaptation on Language Embedding](https://mlsanity.com/paper/2206.03484/) (2022)
- [Automated Detection of Label Errors in Semantic Segmentation Datasets via Deep Learning and Uncertainty Quantification](https://mlsanity.com/paper/2207.06104/) (2022)
- [CAT: Exploiting Inter-Class Dynamics for Domain Adaptive Object Detection](https://mlsanity.com/paper/2403.19278/) (2024)
- [Universal-RCNN: Universal Object Detector via Transferable Graph R-CNN](https://mlsanity.com/paper/2002.07417/) (2020)
- [ScaleDet: A Scalable Multi-Dataset Object Detector](https://mlsanity.com/paper/2306.04849/) (2023)
- [LMSeg: Language-guided Multi-dataset Segmentation](https://mlsanity.com/paper/2302.13495/) (2023)
- [Multi-domain semantic segmentation with overlapping labels](https://mlsanity.com/paper/2108.11224/) (2021)
- [Plain-Det: A Plain Multi-Dataset Object Detector](https://mlsanity.com/paper/2407.10083/) (2024)
- [Automatic universal taxonomies for multi-domain semantic segmentation](https://mlsanity.com/paper/2207.08445/) (2022)

## Built on by (most cited first, 3 of 3)

- [Generalization Under Scrutiny: Cross-Domain Detection Progresses, Pitfalls, and Persistent Challenges](https://mlsanity.com/paper/2604.08230/) (2026)
- [Improving Layout Representation Learning Across Inconsistently Annotated Datasets via Agentic Harmonization](https://mlsanity.com/paper/2604.11042/) (2026)
- [The MODA General Attribute Suite: A Four-Track Evaluation Benchmark for Fashion Attribute Extraction](https://mlsanity.com/paper/2609.13279/) (2026)

## Abstract (arXiv)

Combining multiple object detection datasets offers a path to improved generalisation but is hindered by inconsistencies in class semantics and bounding box annotations. Some methods to address this assume shared label taxonomies and address only spatial inconsistencies; others require manual relabelling, or produce a unified label space, which may be unsuitable when a fixed target label space is required. We propose Label-Aligned Transfer (LAT), a label transfer framework that systematically projects annotations from diverse source datasets into the label space of a target dataset. LAT begins by training dataset-specific detectors to generate pseudo-labels, which are then combined with ground-truth annotations via a Privileged Proposal Generator (PPG) that replaces the region proposal network in two-stage detectors. To further refine region features, a Semantic Feature Fusion (SFF) module injects class-aware context and features from overlapping proposals using a confidence-weighted attention mechanism. This pipeline preserves dataset-specific annotation granularity while enabling many-to-one label space transfer across heterogeneous datasets, resulting in a semantically and spatially aligned representation suitable for training a downstream detector. LAT thus jointly addresses both class-level misalignments and bounding box inconsistencies without relying on shared label spaces or manual annotations. Across multiple benchmarks, LAT demonstrates consistent improvements in target-domain detection performance, achieving gains of up to +4.8AP over semi-supervised baselines.