mlsanityData to 14 Oct 2026

CLIP-MG: Guiding Semantic Attention with Skeletal Pose Features and RGB Data for Micro-Gesture Recognition on the iMiGUE Dataset

arXiv 2506.16385 ↗First posted 19 Jun 2025cs.CV cs.AI cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Micro-gesture recognition is a challenging task in affective computing due to the subtle, involuntary nature of the gestures and their low movement amplitude. In this paper, we introduce a Pose-Guided Semantics-Aware CLIP-based architecture, or CLIP for Micro-Gesture recognition (CLIP-MG), a modified CLIP model tailored for micro-gesture classification on the iMiGUE dataset. CLIP-MG integrates human pose (skeleton) information into the CLIP-based recognition pipeline through pose-guided semantic query generation and a gated multi-modal fusion mechanism. The proposed model achieves a Top-1 accuracy of 61.82%. These results demonstrate both the potential of our approach and the remaining difficulty in fully adapting vision-language models like CLIP for micro-gesture recognition.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 17 of its 33 references that are in the map. First posted on arXiv 19 Jun 2025.

Builds on

17 of 33 references.

References · 17

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  4. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,209
  5. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,064
  6. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 856
  7. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 809
  8. 2017Temporal Relational Reasoning in Videoscited by 324
  9. 2017Gated Multimodal Units for Information Fusioncited by 248
  10. 2021iMiGUE: An Identity-free Video Dataset for Micro-Gesture Understanding and Emotion Analysiscited by 63
  11. 2024MADTP: Multimodal Alignment-Guided Dynamic Token Pruning for Accelerating Vision-Language Transformercited by 54
  12. 2025TokenCarve: Information-Preserving Visual Token Compression in Multimodal Large Language Modelscited by 19
5 more
  1. 2024Prototype Learning for Micro-gesture Classificationcited by 18
  2. 2025Token Sequence Compression for Efficient Multimodal Computingcited by 6
  3. 2024Integrating Large Language Models into a Tri-Modal Architecture for Automated Depression Classification on the DAIC-WOZcited by 4
  4. 2025Task-Related Token Compression in Multimodal Large Language Models from an Explainability Perspectivecited by 2
  5. 2025Speed-up of Vision Transformer Models by Attention-aware Token Filteringcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2026iMiGUE-3K: A Large-Scale Benchmark for Micro-Gesture Analysis with Self-Supervised Learningcited by 3
  2. 2025CLARGA: Multimodal Graph Representation Learning over Arbitrary Sets of Modalitiescited by 0
  3. 2026FG-SGL: Fine-Grained Semantic Guidance Learning via Motion Process Decomposition for Micro-Gesture Recognitioncited by 0
  4. 2026Zero-MELO: Test-Time Evidence Calibration with Multimodal LLMs for Zero-Shot Micro-Gesture Recognitioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025MSF-Mamba: Motion-aware State Fusion Mamba for Efficient Micro-Gesture Recognition
  2. 2026B-MoE: A Body-Part-Aware Mixture-of-Experts "All Parts Matter" Approach to Micro-Action Recognition
  3. 2024Robotic-CLIP: Fine-tuning CLIP on Action Data for Robotic Applications
  4. 2023GestureDiffuCLIP: Gesture Diffusion Model with CLIP Latents
  5. 2026GMoT: Gated Motion-Aware Tokenization for Fine-Grained Micro-Gesture Video Reasoning with Multimodal LLMs
  6. 2025Advancing Vision-based Human Action Recognition: Exploring Vision-Language CLIP Model for Generalisation in Domain-Independent Tasks
  7. 2024Hi5: Synthetic Data for Inclusive, Robust, Hand Pose Estimation
  8. 2025microCLIP: Unsupervised CLIP Adaptation via Coarse-Fine Token Fusion for Fine-Grained Image Classification