mlsanityData to 7 Oct 2026

Open Vocabulary Multi-Label Classification with Dual-Modal Decoder on Aligned Visual-Textual Features

arXiv 2208.09562 ↗First posted 19 Aug 2022cs.CV cs.AI cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

In computer vision, multi-label recognition are important tasks with many real-world applications, but classifying previously unseen labels remains a significant challenge. In this paper, we propose a novel algorithm, Aligned Dual moDality ClaSsifier (ADDS), which includes a Dual-Modal decoder (DM-decoder) with alignment between visual and textual features, for open-vocabulary multi-label classification tasks. Then we design a simple and yet effective method called Pyramid-Forwarding to enhance the performance for inputs with high resolutions. Moreover, the Selective Language Supervision is applied to further enhance the model performance. Extensive experiments conducted on several standard benchmarks, NUS-WIDE, ImageNet-1k, ImageNet-21k, and MS-COCO, demonstrate that our approach significantly outperforms previous methods and provides state-of-the-art performance for open-vocabulary multi-label classification, conventional multi-label classification and an extreme case called single-to-multi label classification where models trained on single-label datasets (ImageNet-1k, ImageNet-21k) are tested on multi-label ones (MS-COCO and NUS-WIDE).

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 40 of its 50 references that are in the map. First posted on arXiv 19 Aug 2022.

Builds on

40 of 50 references.

References · 40

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  7. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,291
  8. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  9. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 903
  10. 2021ImageNet-21K Pretraining for the Massescited by 607
  11. 2021Scaling Open-Vocabulary Image Segmentation with Image-Level Labelscited by 471
  12. 2019Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-trainingcited by 451
28 more
  1. 2021SLIP: Self-supervision meets Language-Image Pre-trainingcited by 434
  2. 2020Open-Vocabulary Object Detection Using Captionscited by 427
  3. 2020Asymmetric Loss For Multi-Label Classificationcited by 379
  4. 2019Multi-Label Image Recognition with Graph Convolutional Networkscited by 320
  5. 2022Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Modelcited by 317
  6. 2018Bilinear Attention Networkscited by 277
  7. 2022Vision-Language Pre-Training with Triple Contrastive Learningcited by 212
  8. 2015Label-Embedding for Image Classificationcited by 164
  9. 2018Deep learning in agriculture: A surveycited by 160
  10. 2013Zero-Shot Learning by Convex Combination of Semantic Embeddingscited by 135
  11. 2021Query2Label: A Simple Transformer Way to Multi-Label Classificationcited by 126
  12. 2019Learning Semantic-Specific Graph Representation for Multi-Label Image Recognitioncited by 116
  13. 2017Multi-label Image Recognition by Recurrently Discovering Attentional Regionscited by 83
  14. 2020Attention-Driven Dynamic Graph Convolutional Network for Multi-Label Image Recognitioncited by 80
  15. 2019Cross-Modality Attention with Semantic Graph Embedding for Multi-Label Classificationcited by 78
  16. 2021ML-Decoder: Scalable and Versatile Classification Headcited by 78
  17. 2017Multi-Label Zero-Shot Learning with Structured Knowledge Graphscited by 72
  18. 2021Open Vocabulary Object Detection with Pseudo Bounding-Box Labelscited by 71
  19. 2020Learning to Discover Multi-Class Attentional Regions for Multi-Label Image Recognitioncited by 50
  20. 2015Exploit Bounding Box Annotations for Multi-label Object Recognitioncited by 39
  21. 2018Multi-Label Image Classification via Knowledge Distillation from Weakly-Supervised Detectioncited by 37
  22. 2021Discriminative Region-based Multi-Label Zero-Shot Learningcited by 27
  23. 2021Semantic Diversity Learning for Zero-Shot Multi-label Classificationcited by 24
  24. 2016Fast Zero-Shot Image Taggingcited by 21
  25. 2021Generative Multi-Label Zero-Shot Learningcited by 19
  26. 2020Multi-label Few/Zero-shot Learning with Knowledge Aggregated from Multiple Label Graphscited by 18
  27. 2022Localized Vision-Language Matching for Open-vocabulary Object Detectioncited by 17
  28. 2019Multi-Label Graph Convolutional Network Representation Learningcited by 10

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 4.

6 papers.

  1. 2024CatLIP: CLIP-level Visual Recognition Accuracy with 2.7x Faster Pre-training on Web-scale Image-Text Datacited by 10
  2. 2025Recover and Match: Open-Vocabulary Multi-Label Recognition through Knowledge-Constrained Optimal Transportcited by 6
  3. 2024SSPA: Split-and-Synthesize Prompting with Gated Alignments for Multi-Label Image Recognitioncited by 2
  4. 2025Diffusion-Based Cross-Modal Feature Extraction for Multi-Label Classificationcited by 2
  5. 2025PatchDEMUX: A Certifiably Robust Framework for Multi-label Classifiers Against Adversarial Patchescited by 1
  6. 2025OTTER: Open-Tagging via Text-Image Representation for Multi-modal Understandingcited by 0