# Action Classification via Concepts and Attributes

arXiv: https://arxiv.org/abs/1605.07824 · HTML: https://mlsanity.com/paper/1605.07824/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 May 2016.

## Built on by (most cited first, 5 of 5)

- [SR-GNN: Spatial Relation-aware Graph Neural Network for Fine-Grained Image Categorization](https://mlsanity.com/paper/2209.02109/) (2022)
- [Attend and Guide (AG-Net): A Keypoints-driven Attention-based Deep Network for Image Recognition](https://mlsanity.com/paper/2110.12183/) (2021)
- [Multi-Expert Human Action Recognition with Hierarchical Super-Class Learning](https://mlsanity.com/paper/2112.07015/) (2021)
- [Regional Attention Network (RAN) for Head Pose and Fine-grained Gesture Recognition](https://mlsanity.com/paper/2101.06634/) (2021)
- [How Do You Do It? Fine-Grained Action Understanding with Pseudo-Adverbs](https://mlsanity.com/paper/2203.12344/) (2022)

## Abstract (arXiv)

Classes in natural images tend to follow long tail distributions. This is problematic when there are insufficient training examples for rare classes. This effect is emphasized in compound classes, involving the conjunction of several concepts, such as those appearing in action-recognition datasets. In this paper, we propose to address this issue by learning how to utilize common visual concepts which are readily available. We detect the presence of prominent concepts in images and use them to infer the target labels instead of using visual features directly, combining tools from vision and natural-language processing. We validate our method on the recently introduced HICO dataset reaching a mAP of 31.54% and on the Stanford-40 Actions dataset, where the proposed method outperforms that obtained by direct visual features, obtaining an accuracy 83.12%. Moreover, the method provides for each class a semantically meaningful list of keywords and relevant image regions relating it to its constituent concepts.