# Orderless Recurrent Models for Multi-label Classification

arXiv: https://arxiv.org/abs/1911.09996 · HTML: https://mlsanity.com/paper/1911.09996/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Nov 2019.

## Built on by (most cited first, 30 of 46)

- [DualCoOp: Fast Adaptation to Multi-Label Recognition with Limited Annotations](https://mlsanity.com/paper/2206.09541/) (2022)
- [Transformer-based Dual Relation Graph for Multi-label Image Recognition](https://mlsanity.com/paper/2110.04722/) (2021)
- [Large Loss Matters in Weakly Supervised Multi-Label Classification](https://mlsanity.com/paper/2206.03740/) (2022)
- [Modeling Multi-Label Action Dependencies for Temporal Action Localization](https://mlsanity.com/paper/2103.03027/) (2021)
- [Discriminative Region-based Multi-Label Zero-Shot Learning](https://mlsanity.com/paper/2108.09301/) (2021)
- [Exploring Structured Semantic Prior for Multi Label Recognition with Incomplete Labels](https://mlsanity.com/paper/2303.13223/) (2023)
- [DualCoOp++: Fast and Effective Adaptation to Multi-Label Recognition with Limited Annotations](https://mlsanity.com/paper/2308.01890/) (2023)
- [Towards Unbiased Multi-label Zero-Shot Learning with Pyramid and Semantic Attention](https://mlsanity.com/paper/2203.03483/) (2022)
- [PatchCT: Aligning Patch Set and Label Set with Conditional Transport for Multi-Label Image Classification](https://mlsanity.com/paper/2307.09066/) (2023)
- [Generative Multi-Label Zero-Shot Learning](https://mlsanity.com/paper/2101.11606/) (2021)
- [Deep Learning for Multi-Label Learning: A Comprehensive Survey](https://mlsanity.com/paper/2401.16549/) (2024)
- [Sewer-ML: A Multi-Label Sewer Defect Classification Dataset and Benchmark](https://mlsanity.com/paper/2103.10895/) (2021)
- [Inferring Prototypes for Multi-Label Few-Shot Image Classification with Word Vector Guided Attention](https://mlsanity.com/paper/2112.01037/) (2021)
- [Test-Time Backdoor Detection for Object Detection Models](https://mlsanity.com/paper/2503.15293/) (2025)
- [From Efficient Multimodal Models to World Models: A Survey](https://mlsanity.com/paper/2407.00118/) (2024)
- [GM-MLIC: Graph Matching based Multi-Label Image Classification](https://mlsanity.com/paper/2104.14762/) (2021)
- [Multi-task Learning of Histology and Molecular Markers for Classifying Diffuse Glioma](https://mlsanity.com/paper/2303.14845/) (2023)
- [Graph Attention Transformer Network for Multi-Label Image Classification](https://mlsanity.com/paper/2203.04049/) (2022)
- [Correlation-Aware Graph Convolutional Networks for Multi-Label Node Classification](https://mlsanity.com/paper/2411.17350/) (2024)
- [NeuralTailor: Reconstructing Sewing Pattern Structures from 3D Point Clouds of Garments](https://mlsanity.com/paper/2201.13063/) (2022)
- [End-to-End Supervised Multilabel Contrastive Learning](https://mlsanity.com/paper/2307.03967/) (2023)
- [Semantic-Aware Graph Matching Mechanism for Multi-Label Image Recognition](https://mlsanity.com/paper/2304.11275/) (2023)
- [Semantic Representation and Dependency Learning for Multi-Label Image Recognition](https://mlsanity.com/paper/2204.03795/) (2022)
- [Improving Multi-label Recognition using Class Co-Occurrence Probabilities](https://mlsanity.com/paper/2404.16193/) (2024)
- [Learning Discriminative Representations for Multi-Label Image Recognition](https://mlsanity.com/paper/2107.11159/) (2021)
- [PersonalTailor: Personalizing 2D Pattern Design from 3D Garment Point Clouds](https://mlsanity.com/paper/2303.09695/) (2023)
- [Text as Image: Learning Transferable Adapter for Multi-Label Classification](https://mlsanity.com/paper/2312.04160/) (2023)
- [ProbMCL: Simple Probabilistic Contrastive Learning for Multi-label Visual Classification](https://mlsanity.com/paper/2401.01448/) (2024)
- [HSVLT: Hierarchical Scale-Aware Vision-Language Transformer for Multi-Label Image Classification](https://mlsanity.com/paper/2407.16244/) (2024)
- [Advances in Set Function Learning: A Survey of Techniques and Applications](https://mlsanity.com/paper/2501.14991/) (2025)

## Abstract (arXiv)

Recurrent neural networks (RNN) are popular for many computer vision tasks, including multi-label classification. Since RNNs produce sequential outputs, labels need to be ordered for the multi-label classification task. Current approaches sort labels according to their frequency, typically ordering them in either rare-first or frequent-first. These imposed orderings do not take into account that the natural order to generate the labels can change for each image, e.g. first the dominant object before summing up the smaller objects in the image. Therefore, in this paper, we propose ways to dynamically order the ground truth labels with the predicted label sequence. This allows for the faster training of more optimal LSTM models for multi-label classification. Analysis evidences that our method does not suffer from duplicate generation, something which is common for other models. Furthermore, it outperforms other CNN-RNN models, and we show that a standard architecture of an image encoder and language decoder trained with our proposed loss obtains the state-of-the-art results on the challenging MS-COCO, WIDER Attribute and PA-100K and competitive results on NUS-WIDE.