# Deep Imbalanced Attribute Classification using Visual Attention Aggregation

arXiv: https://arxiv.org/abs/1807.03903 · HTML: https://mlsanity.com/paper/1807.03903/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jul 2018.

## Built on by (most cited first, 30 of 56)

- [Contrastive Learning based Hybrid Networks for Long-Tailed Image Classification](https://mlsanity.com/paper/2103.14267/) (2021)
- [Semantically Self-Aligned Network for Text-to-Image Part-aware Person Re-identification](https://mlsanity.com/paper/2107.12666/) (2021)
- [DualCoOp: Fast Adaptation to Multi-Label Recognition with Limited Annotations](https://mlsanity.com/paper/2206.09541/) (2022)
- [The Majority Can Help The Minority: Context-rich Minority Oversampling for Long-tailed Classification](https://mlsanity.com/paper/2112.00412/) (2021)
- [Learning to Predict Visual Attributes in the Wild](https://mlsanity.com/paper/2106.09707/) (2021)
- [Beyond Appearance: a Semantic Controllable Self-Supervised Learning Framework for Human-Centric Visual Tasks](https://mlsanity.com/paper/2303.17602/) (2023)
- [Residual Attention: A Simple but Effective Method for Multi-Label Recognition](https://mlsanity.com/paper/2108.02456/) (2021)
- [Self-Damaging Contrastive Learning](https://mlsanity.com/paper/2106.02990/) (2021)
- [Fill-Up: Balancing Long-Tailed Data with Generative Models](https://mlsanity.com/paper/2306.07200/) (2023)
- [Spatial and Semantic Consistency Regularizations for Pedestrian Attribute Recognition](https://mlsanity.com/paper/2109.05686/) (2021)
- [Rethinking of Pedestrian Attribute Recognition: A Reliable Evaluation under Zero-Shot Pedestrian Identity Setting](https://mlsanity.com/paper/2107.03576/) (2021)
- [Pedestrian Attribute Recognition via CLIP based Prompt Vision-Language Fusion](https://mlsanity.com/paper/2312.10692/) (2023)
- [Balanced Knowledge Distillation for Long-tailed Learning](https://mlsanity.com/paper/2104.10510/) (2021)
- [Label2Label: A Language Modeling Framework for Multi-Attribute Learning](https://mlsanity.com/paper/2207.08677/) (2022)
- [DualCoOp++: Fast and Effective Adaptation to Multi-Label Recognition with Limited Annotations](https://mlsanity.com/paper/2308.01890/) (2023)
- [Cross-Domain Empirical Risk Minimization for Unbiased Long-tailed Classification](https://mlsanity.com/paper/2112.14380/) (2021)
- [PARFormer: Transformer-based Multi-Task Network for Pedestrian Attribute Recognition](https://mlsanity.com/paper/2304.07230/) (2023)
- [Hierarchical Visual Primitive Experts for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2308.04016/) (2023)
- [SequencePAR: Understanding Pedestrian Attributes via A Sequence Generation Paradigm](https://mlsanity.com/paper/2312.01640/) (2023)
- [UPAR: Unified Pedestrian Attribute Recognition and Person Retrieval](https://mlsanity.com/paper/2209.02522/) (2022)
- [A Solution to Co-occurrence Bias: Attributes Disentanglement via Mutual Information Minimization for Pedestrian Attribute Recognition](https://mlsanity.com/paper/2307.15252/) (2023)
- [An Empirical Study of Mamba-based Pedestrian Attribute Recognition](https://mlsanity.com/paper/2407.10374/) (2024)
- [LCReg: Long-Tailed Image Classification with Latent Categories based Recognition](https://mlsanity.com/paper/2309.07186/) (2023)
- [Pedestrian Attribute Recognition in Video Surveillance Scenarios Based on View-attribute Attention Localization](https://mlsanity.com/paper/2106.06485/) (2021)
- [Data-free Multi-label Image Recognition via LLM-powered Prompt Tuning](https://mlsanity.com/paper/2403.01209/) (2024)
- [SSPNet: Scale and Spatial Priors Guided Generalizable and Interpretable Pedestrian Attribute Recognition](https://mlsanity.com/paper/2312.06049/) (2023)
- [How does promoting the minority fraction affect generalization? A theoretical study of the one-hidden-layer neural network on group imbalance](https://mlsanity.com/paper/2403.07310/) (2024)
- [A Federated Approach to Predicting Emojis in Hindi Tweets](https://mlsanity.com/paper/2211.06401/) (2022)
- [ASDnB: Merging Face with Body Cues For Robust Active Speaker Detection](https://mlsanity.com/paper/2412.08594/) (2024)
- [Attend Who is Weak: Pruning-assisted Medical Image Localization under Sophisticated and Implicit Imbalances](https://mlsanity.com/paper/2212.02675/) (2022)

## Abstract (arXiv)

For many computer vision applications, such as image description and human identification, recognizing the visual attributes of humans is an essential yet challenging problem. Its challenges originate from its multi-label nature, the large underlying class imbalance and the lack of spatial annotations. Existing methods follow either a computer vision approach while failing to account for class imbalance, or explore machine learning solutions, which disregard the spatial and semantic relations that exist in the images. With that in mind, we propose an effective method that extracts and aggregates visual attention masks at different scales. We introduce a loss function to handle class imbalance both at class and at an instance level and further demonstrate that penalizing attention masks with high prediction variance accounts for the weak supervision of the attention mechanism. By identifying and addressing these challenges, we achieve state-of-the-art results with a simple attention mechanism in both PETA and WIDER-Attribute datasets without additional context or side information.