# Multi-Label Activity Recognition using Activity-specific Features and Activity Correlations

arXiv: https://arxiv.org/abs/2009.07420 · HTML: https://mlsanity.com/paper/2009.07420/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Sep 2020.

## Built on by (most cited first, 15 of 15)

- [TubeR: Tubelet Transformer for Video Action Detection](https://mlsanity.com/paper/2104.00969/) (2021)
- [Open Set Action Recognition via Multi-Label Evidential Learning](https://mlsanity.com/paper/2303.12698/) (2023)
- [Detector-Free Weakly Supervised Group Activity Recognition](https://mlsanity.com/paper/2204.02139/) (2022)
- [SPARTAN: Self-supervised Spatiotemporal Transformers Approach to Group Activity Recognition](https://mlsanity.com/paper/2303.12149/) (2023)
- [SoGAR: Self-supervised Spatiotemporal Attention-based Social Group Activity Recognition](https://mlsanity.com/paper/2305.06310/) (2023)
- [Actor-agnostic Multi-label Action Recognition with Multi-modal Query](https://mlsanity.com/paper/2307.10763/) (2023)
- [Action Selection Learning for Multi-label Multi-view Action Recognition](https://mlsanity.com/paper/2410.03302/) (2024)
- [REACT: Recognize Every Action Everywhere All At Once](https://mlsanity.com/paper/2312.00188/) (2023)
- [Multi-view Multi-label Fine-grained Emotion Decoding from Human Brain Activity](https://mlsanity.com/paper/2211.02629/) (2022)
- [AdaFPP: Adapt-Focused Bi-Propagating Prototype Learning for Panoramic Activity Recognition](https://mlsanity.com/paper/2405.02538/) (2024)
- [An LLM-Empowered Low-Resolution Vision System for On-Device Human Behavior Understanding](https://mlsanity.com/paper/2505.01743/) (2025)
- [Pushing One Pair of Labels Apart Each Time in Multi-Label Learning: From Single Positive to Full Labels](https://mlsanity.com/paper/2302.14695/) (2023)
- [The impact of Compositionality in Zero-shot Multi-label action recognition for Object-based tasks](https://mlsanity.com/paper/2405.08695/) (2024)
- [Salient Temporal Encoding for Dynamic Scene Graph Generation](https://mlsanity.com/paper/2503.14524/) (2025)
- [THOR: Thermal-guided Hand-Object Reasoning via Adaptive Vision Sampling](https://mlsanity.com/paper/2507.06442/) (2025)

## Abstract (arXiv)

Multi-label activity recognition is designed for recognizing multiple activities that are performed simultaneously or sequentially in each video. Most recent activity recognition networks focus on single-activities, that assume only one activity in each video. These networks extract shared features for all the activities, which are not designed for multi-label activities. We introduce an approach to multi-label activity recognition that extracts independent feature descriptors for each activity and learns activity correlations. This structure can be trained end-to-end and plugged into any existing network structures for video classification. Our method outperformed state-of-the-art approaches on four multi-label activity recognition datasets. To better understand the activity-specific features that the system generated, we visualized these activity-specific features in the Charades dataset.