# Action Selection Learning for Multi-label Multi-view Action Recognition

arXiv: https://arxiv.org/abs/2410.03302 · HTML: https://mlsanity.com/paper/2410.03302/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 51 references that are in the map.
- First posted on arXiv 4 Oct 2024.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases](https://mlsanity.com/paper/2103.10697/) (2021)
- [Human Action Recognition and Prediction: A Survey](https://mlsanity.com/paper/1806.11230/) (2018)
- [Multiple Instance Learning: A Survey of Problem Characteristics and Applications](https://mlsanity.com/paper/1612.03365/) (2016)
- [Human Action Recognition from Various Data Modalities: A Review](https://mlsanity.com/paper/2012.11866/) (2020)
- [Query2Label: A Simple Transformer Way to Multi-Label Classification](https://mlsanity.com/paper/2107.10834/) (2021)
- [Every Moment Counts: Dense Detailed Labeling of Actions in Complex Videos](https://mlsanity.com/paper/1507.05738/) (2015)
- [View Adaptive Neural Networks for High Performance Skeleton-based Human Action Recognition](https://mlsanity.com/paper/1804.07453/) (2018)
- [Deep Analysis of CNN-based Spatio-temporal Representations for Action Recognition](https://mlsanity.com/paper/2010.11757/) (2020)
- [Quo Vadis, Skeleton Action Recognition ?](https://mlsanity.com/paper/2007.02072/) (2020)
- [AdaSGN: Adapting Joint Number and Model Size for Efficient Skeleton-Based Action Recognition](https://mlsanity.com/paper/2103.11770/) (2021)
- [Weakly Supervised Action Selection Learning in Video](https://mlsanity.com/paper/2105.02439/) (2021)
- [Multi-Label Activity Recognition using Activity-specific Features and Activity Correlations](https://mlsanity.com/paper/2009.07420/) (2020)
- [Multi-view and Multi-modal Event Detection Utilizing Transformer-based Multi-sensor fusion](https://mlsanity.com/paper/2202.09124/) (2022)
- [Actor-agnostic Multi-label Action Recognition with Multi-modal Query](https://mlsanity.com/paper/2307.10763/) (2023)
- [Collaborative Attention Mechanism for Multi-View Action Recognition](https://mlsanity.com/paper/2009.06599/) (2020)
- [Guided Masked Self-Distillation Modeling for Distributed Multimedia Sensor Event Analysis](https://mlsanity.com/paper/2404.08264/) (2024)

## Built on by (most cited first, 6 of 6)

- [MultiSensor-Home: A Wide-area Multi-modal Multi-view Dataset for Action Recognition and Transformer-based Sensor Fusion](https://mlsanity.com/paper/2504.02287/) (2025)
- [Trunk-branch Contrastive Network with Multi-view Deformable Aggregation for Multi-view Action Recognition](https://mlsanity.com/paper/2502.16493/) (2025)
- [Multi Camera Connected Vision System with Multi View Analytics: A Comprehensive Survey](https://mlsanity.com/paper/2510.09731/) (2025)
- [View-aware Cross-modal Distillation for Multi-view Action Recognition](https://mlsanity.com/paper/2511.12870/) (2025)
- [MultiTSF: Transformer-based Sensor Fusion for Human-Centric Multi-view and Multi-modal Action Recognition](https://mlsanity.com/paper/2504.02279/) (2025)
- [Moving Beyond More Views: Redundancy-Aware Ego-Exo Fusion for Proficiency Estimation](https://mlsanity.com/paper/2608.25736/) (2026)

## Abstract (arXiv)

Multi-label multi-view action recognition aims to recognize multiple concurrent or sequential actions from untrimmed videos captured by multiple cameras. Existing work has focused on multi-view action recognition in a narrow area with strong labels available, where the onset and offset of each action are labeled at the frame-level. This study focuses on real-world scenarios where cameras are distributed to capture a wide-range area with only weak labels available at the video-level. We propose the method named Multi-view Action Selection Learning (MultiASL), which leverages action selection learning to enhance view fusion by selecting the most useful information from different viewpoints. The proposed method includes a Multi-view Spatial-Temporal Transformer video encoder to extract spatial and temporal features from multi-viewpoint videos. Action Selection Learning is employed at the frame-level, using pseudo ground-truth obtained from weak labels at the video-level, to identify the most relevant frames for action recognition. Experiments in a real-world office environment using the MM-Office dataset demonstrate the superior performance of the proposed method compared to existing methods. The source code is available at https://github.com/thanhhff/MultiASL/.