# SoftRerank: Hierarchical Soft Fusion with Candidate-Label Reranking for Long-Tailed Micro-Action Recognition

arXiv: https://arxiv.org/abs/2609.08221 · HTML: https://mlsanity.com/paper/2609.08221/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 36 references that are in the map.
- First posted on arXiv 8 Sep 2026.

## Builds on (most cited first, 19 of 19)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modeling](https://mlsanity.com/paper/2501.12386/) (2025)
- [Co-occurrence Feature Learning from Skeleton Data for Action Recognition and Detection with Hierarchical Aggregation](https://mlsanity.com/paper/1804.06055/) (2018)
- [PYSKL: Towards Good Practices for Skeleton Action Recognition](https://mlsanity.com/paper/2205.09443/) (2022)
- [Dynamic GCN: Context-enriched Topology Learning for Skeleton-based Action Recognition](https://mlsanity.com/paper/2007.14690/) (2020)
- [Skeleton-based Action Recognition with Convolutional Neural Networks](https://mlsanity.com/paper/1704.07595/) (2017)
- [Revealing Key Details to See Differences: A Novel Prototypical Perspective for Skeleton-based Action Recognition](https://mlsanity.com/paper/2411.18941/) (2024)
- [Joint Skeletal and Semantic Embedding Loss for Micro-gesture Classification](https://mlsanity.com/paper/2307.10624/) (2023)
- [Prototypical Calibrating Ambiguous Samples for Micro-Action Recognition](https://mlsanity.com/paper/2412.14719/) (2024)
- [MMAD: Multi-label Micro-Action Detection in Videos](https://mlsanity.com/paper/2407.05311/) (2024)
- [Prototype Learning for Micro-gesture Classification](https://mlsanity.com/paper/2408.03097/) (2024)
- [MM-Gesture: Towards Precise Micro-Gesture Recognition through Multimodal Fusion](https://mlsanity.com/paper/2507.08344/) (2025)
- [MotionSight: Boosting Fine-Grained Motion Understanding in Multimodal LLMs](https://mlsanity.com/paper/2506.01674/) (2025)
- [SeFAR: Semi-supervised Fine-grained Action Recognition with Temporal Perturbation and Learning Stabilization](https://mlsanity.com/paper/2501.01245/) (2025)
- [Action-Agnostic Point-Level Supervision for Temporal Action Detection](https://mlsanity.com/paper/2412.21205/) (2024)
- [Self-supervised Learning Matters: A Simple Ensemble Solution for Micro-Gesture Recognition](https://mlsanity.com/paper/2606.09261/) (2026)
- [Rethinking the Role of Feature Engineering and Learning Strategies in Few-Shot Hidden Emotion Recognition](https://mlsanity.com/paper/2606.31249/) (2026)
- [MAC 2026: Advancing Micro-Action Analysis Towards Fine-Grained Understanding](https://mlsanity.com/paper/2607.16284/) (2026)

## Abstract (arXiv)

Micro-actions are subtle, low-intensity non-verbal behaviors that provide cues to fine-grained human states, including emotions and intentions. Recognizing them remains difficult because they are brief, contain weak visual changes, and often exhibit similar motion patterns across categories. This paper addresses these challenges with a fine-grained micro-action recognition method that combines full fine-tuning of InternVideo2.5, hierarchical soft fusion, and a lightweight candidate-label reranker. For the long-tailed label distribution in MA-52, we use class-balanced sampling and inverse-frequency reweighting to reduce the effect of frequent classes during training. We fine-tune InternVideo2.5 end to end and attach coarse and group-conditional fine-grained classification heads to the shared video representation, improving the consistency between coarse and fine predictions. For ambiguous samples, the candidate-label reranker uses hard samples and video-label matching to focus on easily confused fine-grained actions. Experiments validate the proposed method, which achieves a 79.99% F1-mean on MA-52 and ranks first in the 3rd Micro-Action Analysis Grand Challenge at ACM Multimedia 2026.