# Vision-language Assisted Attribute Learning

arXiv: https://arxiv.org/abs/2312.07009 · HTML: https://mlsanity.com/paper/2312.07009/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 18 references that are in the map.
- First posted on arXiv 12 Dec 2023.

## Builds on (most cited first, 8 of 8)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Learning a Deep ConvNet for Multi-label Classification with Partial Labels](https://mlsanity.com/paper/1902.09720/) (2019)
- [From Captions to Visual Concepts and Back](https://mlsanity.com/paper/1411.4952/) (2014)
- [Open World Compositional Zero-Shot Learning](https://mlsanity.com/paper/2101.12609/) (2021)
- [Learning to Predict Visual Attributes in the Wild](https://mlsanity.com/paper/2106.09707/) (2021)
- [AniGAN: Style-Guided Generative Adversarial Networks for Unsupervised Anime Face Generation](https://mlsanity.com/paper/2102.12593/) (2021)
- [Multi-label Classification with Partial Annotations using Class-aware Selective Loss](https://mlsanity.com/paper/2110.10955/) (2021)

## Abstract (arXiv)

Attribute labeling at large scale is typically incomplete and partial, posing significant challenges to model optimization. Existing attribute learning methods often treat the missing labels as negative or simply ignore them all during training, either of which could hamper the model performance to a great extent. To overcome these limitations, in this paper we leverage the available vision-language knowledge to explicitly disclose the missing labels for enhancing model learning. Given an image, we predict the likelihood of each missing attribute label assisted by an off-the-shelf vision-language model, and randomly select to ignore those with high scores in training. Our strategy strikes a good balance between fully ignoring and negatifying the missing labels, as these high scores are found to be informative on revealing label ambiguity. Extensive experiments show that our proposed vision-language assisted loss can achieve state-of-the-art performance on the newly cleaned VAW dataset. Qualitative evaluation demonstrates the ability of the proposed method in predicting more complete attributes.