# Motion Reinforces Appearance: RGB-Skeleton Gated Residual Fusion for Micro-Gesture Online Recognition

arXiv: https://arxiv.org/abs/2606.11645 · HTML: https://mlsanity.com/paper/2606.11645/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 44 references that are in the map.
- First posted on arXiv 10 Jun 2026.

## Builds on (most cited first, 30 of 30)

- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression](https://mlsanity.com/paper/1911.08287/) (2019)
- [BMN: Boundary-Matching Network for Temporal Action Proposal Generation](https://mlsanity.com/paper/1907.09702/) (2019)
- [Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2107.12213/) (2021)
- [ActionFormer: Localizing Moments of Actions with Transformers](https://mlsanity.com/paper/2202.07925/) (2022)
- [Revisiting Skeleton-based Action Recognition](https://mlsanity.com/paper/2104.13586/) (2021)
- [G-TAD: Sub-Graph Localization for Temporal Action Detection](https://mlsanity.com/paper/1911.11462/) (2019)
- [Learning Salient Boundary Feature for Anchor-free Temporal Action Localization](https://mlsanity.com/paper/2103.13137/) (2021)
- [TriDet: Temporal Action Detection with Relative Boundary Modeling](https://mlsanity.com/paper/2303.07347/) (2023)
- [iMiGUE: An Identity-free Video Dataset for Micro-Gesture Understanding and Emotion Analysis](https://mlsanity.com/paper/2107.00285/) (2021)
- [Benchmarking Micro-action Recognition: Dataset, Methods, and Applications](https://mlsanity.com/paper/2403.05234/) (2024)
- [Survey on Emotional Body Gesture Recognition](https://mlsanity.com/paper/1801.07481/) (2018)
- [Joint Skeletal and Semantic Embedding Loss for Micro-gesture Classification](https://mlsanity.com/paper/2307.10624/) (2023)
- [PointTAD: Multi-Label Temporal Action Detection with Learnable Query Points](https://mlsanity.com/paper/2210.11035/) (2022)
- [Prototypical Calibrating Ambiguous Samples for Micro-Action Recognition](https://mlsanity.com/paper/2412.14719/) (2024)
- [MMAD: Multi-label Micro-Action Detection in Videos](https://mlsanity.com/paper/2407.05311/) (2024)
- [DyFADet: Dynamic Feature Aggregation for Temporal Action Detection](https://mlsanity.com/paper/2407.03197/) (2024)
- [Group Contextualization for Video Recognition](https://mlsanity.com/paper/2203.09694/) (2022)
- [Micro-gesture Online Recognition using Learnable Query Points](https://mlsanity.com/paper/2407.04490/) (2024)
- [Prototype Learning for Micro-gesture Classification](https://mlsanity.com/paper/2408.03097/) (2024)
- [Data Augmentation for Human Behavior Analysis in Multi-Person Conversations](https://mlsanity.com/paper/2308.01526/) (2023)
- [MM-Gesture: Towards Precise Micro-Gesture Recognition through Multimodal Fusion](https://mlsanity.com/paper/2507.08344/) (2025)
- [Attention in Attention: Modeling Context Correlation for Efficient Video Classification](https://mlsanity.com/paper/2204.09303/) (2022)
- [Online Micro-gesture Recognition Using Data Augmentation and Spatial-Temporal Attention](https://mlsanity.com/paper/2507.09512/) (2025)
- [Motion Matters: Motion-guided Modulation Network for Skeleton-based Micro-Action Recognition](https://mlsanity.com/paper/2507.21977/) (2025)
- [OpenTAD: A Unified Framework and Comprehensive Study of Temporal Action Detection](https://mlsanity.com/paper/2502.20361/) (2025)
- [MA-Bench: Towards Fine-grained Micro-Action Understanding](https://mlsanity.com/paper/2603.26586/) (2026)
- [A Survey on fMRI-based Brain Decoding for Reconstructing Multimodal Stimuli](https://mlsanity.com/paper/2503.15978/) (2025)
- [Spatial-Temporal Decoupled Adapter for Micro-gesture Online Recognition](https://mlsanity.com/paper/2606.07355/) (2026)

## Abstract (arXiv)

Micro-gesture analysis attracts increasing attention for inferring spontaneous emotion from subtle body movements. Micro-gesture online recognition, which localizes and classifies each gesture instance in untrimmed videos, is a core task in the 4th EI-MiGA-IJCAI Challenge. Compared with typical temporal action detection, MGR emphasizes the localization and classification of actions, requiring the model to output the start time, end time, and category of each micro-gesture. Moreover, since micro-gestures are highly spontaneous, relying solely on a single modality makes it difficult to capture the complete and accurate multi-modal cues. In this work, we propose DyFADet+, which extends DyFADet into a dual-stream RGB-skeleton framework. In our model, both modalities are projected into shared multi-scale temporal embeddings and fused through a gated residual module, which adaptively injects skeleton motion into the RGB representation rather than using naive concatenation. Finally, these fused features are decoded by a Dynamic TAD head for online classification and boundary regression. On the SMG dataset, our method achieves an F1 score of 40.88, ranking 2nd in the Micro-gesture Online Recognition track.