# Revising the Problem of Partial Labels from the Perspective of CNNs' Robustness

arXiv: https://arxiv.org/abs/2407.17630 · HTML: https://mlsanity.com/paper/2407.17630/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 38 references that are in the map.
- First posted on arXiv 24 Jul 2024.

## Builds on (most cited first, 15 of 15)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [Object Detection in 20 Years: A Survey](https://mlsanity.com/paper/1905.05055/) (2019)
- [A Survey of Deep Learning Techniques for Autonomous Driving](https://mlsanity.com/paper/1910.07738/) (2019)
- [DeepXplore: Automated Whitebox Testing of Deep Learning Systems](https://mlsanity.com/paper/1705.06640/) (2017)
- [Learning from positive and unlabeled data: a survey](https://mlsanity.com/paper/1811.04820/) (2018)
- [Presence-Only Geographical Priors for Fine-Grained Image Classification](https://mlsanity.com/paper/1906.05272/) (2019)
- [Multi-Label Learning from Single Positive Labels](https://mlsanity.com/paper/2106.09708/) (2021)
- [DeepMutation: Mutation Testing of Deep Learning Systems](https://mlsanity.com/paper/1805.05206/) (2018)
- [Computational discovery of new 2D materials using deep learning generative models](https://mlsanity.com/paper/2012.09314/) (2020)
- [What Are We Really Testing in Mutation Testing for Machine Learning? A Critical Reflection](https://mlsanity.com/paper/2103.01341/) (2021)
- [An Effective Approach for Multi-label Classification with Missing Labels](https://mlsanity.com/paper/2210.13651/) (2022)
- [D-Score: A White-Box Diagnosis Score for CNNs Based on Mutation Operators](https://mlsanity.com/paper/2304.00697/) (2023)
- [Towards Imbalanced Large Scale Multi-label Classification with Partially Annotated Labels](https://mlsanity.com/paper/2308.00166/) (2023)

## Abstract (arXiv)

Convolutional neural networks (CNNs) have gained increasing popularity and versatility in recent decades, finding applications in diverse domains. These remarkable achievements are greatly attributed to the support of extensive datasets with precise labels. However, annotating image datasets is intricate and complex, particularly in the case of multi-label datasets. Hence, the concept of partial-label setting has been proposed to reduce annotation costs, and numerous corresponding solutions have been introduced. The evaluation methods for these existing solutions have been primarily based on accuracy. That is, their performance is assessed by their predictive accuracy on the test set. However, we insist that such an evaluation is insufficient and one-sided. On one hand, since the quality of the test set has not been evaluated, the assessment results are unreliable. On the other hand, the partial-label problem may also be raised by undergoing adversarial attacks. Therefore, incorporating robustness into the evaluation system is crucial. For this purpose, we first propose two attack models to generate multiple partial-label datasets with varying degrees of label missing rates. Subsequently, we introduce a lightweight partial-label solution using pseudo-labeling techniques and a designed loss function. Then, we employ D-Score to analyze both the proposed and existing methods to determine whether they can enhance robustness while improving accuracy. Extensive experimental results demonstrate that while certain methods may improve accuracy, the enhancement in robustness is not significant, and in some cases, it even diminishes.