mlsanityData to 14 Oct 2026

GMoT: Gated Motion-Aware Tokenization for Fine-Grained Micro-Gesture Video Reasoning with Multimodal LLMs

arXiv 2607.16322 ↗First posted 15 Jul 2026cs.CV cs.AIcited by 0

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Micro-gesture recognition demands the detection of fleeting, spatially localized movements that are frequently overwhelmed by dominant static appearances and background noise. While Multimodal Large Language Models (MLLMs) excel at general video understanding, they inherently struggle with subtle kinematics and often rely on static posture priors. To this end, we propose GMoT, a Gated Motion-Aware Tokenization module that explicitly distills sparse kinematic evidence into a compact sequence prior to temporal modeling. GMoT dynamically spotlights action-relevant regions via spatially weighted pooling, extracts adjacent-frame temporal differencing to capture precise motion energy, and adaptively fuses these cues into the visual stream using a conservatively initialized semantic gate. To transition from simple classification to evidence-grounded reasoning, we further introduce a progressive reward-guided policy refinement paradigm, supported by a semi-supervised annotation pipeline that generates anatomically focused captions. Beyond achieving the best Top-1 accuracy among the compared methods on iMiGUE (67.32%) and SMG (73.11%), improving the Qwen3-VL-8B baseline by +6.80 and +3.11 points, our framework introduces Body-Region Grounding (BRG) Recall as an anatomical-grounding proxy conditioned on correct predictions, together with an overlapping-label cross-domain transfer protocol between iMiGUE and SMG. Extensive evaluations demonstrate that our GMoT-augmented model improves in-domain accuracy, retains clear gains under label-preserving corruptions, and improves accuracy-oriented cross-domain transfer under explicit small-split caveats while maintaining high anatomical grounding in its generated rationales.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 31 of its 64 references that are in the map. First posted on arXiv 15 Jul 2026.

Builds on

31 of 64 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Video-R1: Reinforcing Video Reasoning in MLLMs27 Mar
  2. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec

Other references · 29

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2025Qwen2.5-VL Technical Reportcited by 5,587
  6. 2024GPT-4o System Cardcited by 4,588
  7. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  8. 2025Qwen3-VL Technical Reportcited by 2,793
  9. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  10. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
  11. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  12. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
17 more
  1. 2021Video Swin Transformercited by 1,091
  2. 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 355
  3. 2023VideoLLM: Modeling Video Sequence with Large Language Modelscited by 116
  4. 2025LVAgent: Long Video Understanding by Multi-Round Dynamical Collaboration of MLLM Agentscited by 58
  5. 2023Joint Skeletal and Semantic Embedding Loss for Micro-gesture Classificationcited by 25
  6. 2025UniEmo: Unifying Emotional Understanding and Generation with Learnable Expert Queriescited by 18
  7. 2026DeltaVLA: Prior-Guided Vision-Language-Action Models via World Knowledge Variationcited by 10
  8. 2024EALD-MLLM: Emotion Analysis in Long-sequential and De-identity videos with Multi-modal Large Language Modelcited by 8
  9. 2026AffectAgent: Collaborative Multi-Agent Reasoning for Retrieval-Augmented Multimodal Emotion Recognitioncited by 6
  10. 2026OmniOPSD: Rationale-Privileged On-Policy Self-Distillation for Affective Computingcited by 6
  11. 2026AULLM++: Structured-Token-Conditioned Large Language Models for Micro-Expression Action Unit Detectioncited by 4
  12. 2026Navigating the Emotion Tree: Hierarchical Hyperbolic RAG for Multimodal Emotion Recognitioncited by 4
  13. 2025TinyLLaVA-Video: Towards Smaller LMMs for Video Understanding with Group Resamplercited by 2
  14. 2026DFM: Difference Feature Modeling with Text-Guided Gated Contrastive Loss for Remote Sensing Image Change Captioningcited by 2
  15. 2026Reflect-R1: Evidence-Driven Reflection for Self-Correction in Long Video Understandingcited by 2
  16. 2026RSICCLLM: A Multimodal Large Language Model for Remote Sensing Image Change Captioningcited by 2
  17. 2026DeceptionX: From Multimodal Evidence to Explainable Deception Detectioncited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Zero-MELO: Test-Time Evidence Calibration with Multimodal LLMs for Zero-Shot Micro-Gesture Recognition
  2. 2026Recognition-Conditioned Reasoning: A Training-Free Multimodal-LLM Pipeline for Fine-Grained Micro-Action Understanding
  3. 2026KPM-Bench: A Kinematic Parsing Motion Benchmark for Fine-grained Motion-centric Video Understanding
  4. 2026Motion-as-Prompt: Enhancing Motion Reasoning in Multimodal Large Language Models via Motion-Guided Cross-Frame Visual Prompting
  5. 2026MA-Bench: Towards Fine-grained Micro-Action Understanding
  6. 2025FingerCap: Fine-grained Finger-level Hand Motion Captioning
  7. 2025Chain-of-Evidence Multimodal Reasoning for Few-shot Temporal Action Localization
  8. 2025MotionSight: Boosting Fine-Grained Motion Understanding in Multimodal LLMs