# Multi-Expert Human Action Recognition with Hierarchical Super-Class Learning

arXiv: https://arxiv.org/abs/2112.07015 · HTML: https://mlsanity.com/paper/2112.07015/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 56 references that are in the map.
- First posted on arXiv 13 Dec 2021.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Libra R-CNN: Towards Balanced Learning for Object Detection](https://mlsanity.com/paper/1904.02701/) (2019)
- [Detecting and Recognizing Human-Object Interactions](https://mlsanity.com/paper/1704.07333/) (2017)
- [Overcoming Classifier Imbalance for Long-tail Object Detection with Balanced Group Softmax](https://mlsanity.com/paper/2006.10408/) (2020)
- [Smooth Grad-CAM++: An Enhanced Inference Level Visualization Technique for Deep Convolutional Neural Network Models](https://mlsanity.com/paper/1908.01224/) (2019)
- [Learning Human-Object Interaction Detection using Interaction Points](https://mlsanity.com/paper/2003.14023/) (2020)
- [Skeleton-based Action Recognition via Spatial and Temporal Transformer Networks](https://mlsanity.com/paper/2008.07404/) (2020)
- [PaStaNet: Toward Human Activity Knowledge Engine](https://mlsanity.com/paper/2004.00945/) (2020)
- [Detecting Human-Object Interactions with Action Co-occurrence Priors](https://mlsanity.com/paper/2007.08728/) (2020)
- [Prime Sample Attention in Object Detection](https://mlsanity.com/paper/1904.04821/) (2019)
- [Towards Accurate One-Stage Object Detection with AP-Loss](https://mlsanity.com/paper/1904.06373/) (2019)
- [Contextual Action Recognition with R*CNN](https://mlsanity.com/paper/1505.01197/) (2015)
- [DR Loss: Improving Object Detection by Distributional Ranking](https://mlsanity.com/paper/1907.10156/) (2019)
- [Single Image Action Recognition using Semantic Body Part Actions](https://mlsanity.com/paper/1612.04520/) (2016)
- [Generating Positive Bounding Boxes for Balanced Training of Object Detectors](https://mlsanity.com/paper/1909.09777/) (2019)
- [Discriminative feature generation for classification of imbalanced data](https://mlsanity.com/paper/2010.12888/) (2020)
- [Action Classification via Concepts and Attributes](https://mlsanity.com/paper/1605.07824/) (2016)
- [Loss Guided Activation for Action Recognition in Still Images](https://mlsanity.com/paper/1812.04194/) (2018)
- [Ensembles of Deep Neural Networks for Action Recognition in Still Images](https://mlsanity.com/paper/2003.09893/) (2020)

## Built on by (most cited first, 5 of 5)

- [SuperDisco: Super-Class Discovery Improves Visual Recognition for the Long-Tail](https://mlsanity.com/paper/2304.00101/) (2023)
- [Attention-Based Generative Neural Image Compression on Solar Dynamics Observatory](https://mlsanity.com/paper/2210.06478/) (2022)
- [CCFace: Classification Consistency for Low-Resolution Face Recognition](https://mlsanity.com/paper/2308.09230/) (2023)
- [Cross-Hierarchical Bidirectional Consistency Learning for Fine-Grained Visual Classification](https://mlsanity.com/paper/2504.13608/) (2025)
- [Rethinking Crystal Symmetry Prediction: A Decoupled Perspective](https://mlsanity.com/paper/2511.06976/) (2025)

## Abstract (arXiv)

In still image human action recognition, existing studies have mainly leveraged extra bounding box information along with class labels to mitigate the lack of temporal information in still images; however, preparing extra data with manual annotation is time-consuming and also prone to human errors. Moreover, the existing studies have not addressed action recognition with long-tailed distribution. In this paper, we propose a two-phase multi-expert classification method for human action recognition to cope with long-tailed distribution by means of super-class learning and without any extra information. To choose the best configuration for each super-class and characterize inter-class dependency between different action classes, we propose a novel Graph-Based Class Selection (GCS) algorithm. In the proposed approach, a coarse-grained phase selects the most relevant fine-grained experts. Then, the fine-grained experts encode the intricate details within each super-class so that the inter-class variation increases. Extensive experimental evaluations are conducted on various public human action recognition datasets, including Stanford40, Pascal VOC 2012 Action, BU101+, and IHAR datasets. The experimental results demonstrate that the proposed method yields promising improvements. To be more specific, in IHAR, Sanford40, Pascal VOC 2012 Action, and BU101+ benchmarks, the proposed approach outperforms the state-of-the-art studies by 8.92%, 0.41%, 0.66%, and 2.11 % with much less computational cost and without any auxiliary annotation information. Besides, it is proven that in addressing action recognition with long-tailed distribution, the proposed method outperforms its counterparts by a significant margin.