# ChatMotion: A Multimodal Multi-Agent for Human Motion Analysis

arXiv: https://arxiv.org/abs/2502.18180 · HTML: https://mlsanity.com/paper/2502.18180/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 19 of its 44 references that are in the map.
- First posted on arXiv 25 Feb 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)
- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 19 of 19)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://mlsanity.com/paper/2311.10122/) (2023)
- [MVBench: A Comprehensive Multi-modal Video Understanding Benchmark](https://mlsanity.com/paper/2311.17005/) (2023)
- [MotionGPT: Human Motion as a Foreign Language](https://mlsanity.com/paper/2306.14795/) (2023)
- [MotionCLIP: Exposing Human Motion Generation to CLIP Space](https://mlsanity.com/paper/2203.08063/) (2022)
- [The KIT Motion-Language Dataset](https://mlsanity.com/paper/1607.03827/) (2016)
- [We are More than Our Joints: Predicting how 3D Bodies Move](https://mlsanity.com/paper/2012.00619/) (2020)
- [MotionLLM: Understanding Human Behaviors from Human Motions and Videos](https://mlsanity.com/paper/2405.20340/) (2024)
- [Learning a bidirectional mapping between human whole-body motion and natural language using deep recurrent neural networks](https://mlsanity.com/paper/1705.06400/) (2017)
- [Motion Question Answering via Modular Motion Programs](https://mlsanity.com/paper/2305.08953/) (2023)
- [Versatile Multi-Modal Pre-Training for Human-Centric Perception](https://mlsanity.com/paper/2203.13815/) (2022)
- [CHOPS: CHat with custOmer Profile Systems for Customer Service with LLMs](https://mlsanity.com/paper/2404.01343/) (2024)
- [Intrinsic Entropy of Context Length Scaling in LLMs](https://mlsanity.com/paper/2502.01481/) (2025)
- [Graph Canvas for Controllable 3D Scene Generation](https://mlsanity.com/paper/2412.00091/) (2024)
- [Efficient Domain Adaptation via Generative Prior for 3D Infant Pose Estimation](https://mlsanity.com/paper/2311.12043/) (2023)
- [Chain-of-Thought in Large Language Models: Decoding, Projection, and Activation](https://mlsanity.com/paper/2412.03944/) (2024)
- [Reassessing the Role of Chain-of-Thought in Sentiment Analysis: Insights and Limitations](https://mlsanity.com/paper/2501.08641/) (2025)
- [Category-Agnostic Pose Estimation for Point Clouds](https://mlsanity.com/paper/2403.07437/) (2024)
- [Human-Computer Interaction with Adaptable & Adaptive Motion-based Games for Health](https://mlsanity.com/paper/2012.03309/) (2020)

## Built on by (most cited first, 14 of 14)

- [ConeSep: Cone-based Robust Noise-Unlearning Compositional Network for Composed Image Retrieval](https://mlsanity.com/paper/2604.20358/) (2026)
- [Task-Specific Efficiency Analysis: When Small Language Models Outperform Large Language Models](https://mlsanity.com/paper/2603.21389/) (2026)
- [CountLLM: Towards Generalizable Repetitive Action Counting via Large Language Model](https://mlsanity.com/paper/2503.17690/) (2025)
- [RAM: Recover Any 3D Human Motion in-the-Wild](https://mlsanity.com/paper/2603.19929/) (2026)
- [SafeMT: Multi-turn Safety for Multimodal Language Models](https://mlsanity.com/paper/2510.12133/) (2025)
- [Video Understanding: From Geometry and Semantics to Unified Models](https://mlsanity.com/paper/2603.17840/) (2026)
- [Revisiting Image Manipulation Localization under Realistic Manipulation Scenarios](https://mlsanity.com/paper/2509.20006/) (2025)
- [UniMo: Unifying 2D Video and 3D Human Motion with an Autoregressive Framework](https://mlsanity.com/paper/2512.03918/) (2025)
- [Coarse-to-Fine Personalized LLM Impressions for Streamlined Radiology Reports](https://mlsanity.com/paper/2508.15845/) (2025)
- [VigilFormer: Deformable Attention for Video Anomaly Detection with Causal Risk Inference](https://mlsanity.com/paper/2606.14724/) (2026)
- [Bridging Geographic Bias in Urban Streetscape Inference via Lifelong Learning with Visual-Semantic Pivoting](https://mlsanity.com/paper/2606.15055/) (2026)
- [URDF Synthesis from RGB-D Sequences via Differentiable Joint Inference and Energy-Consistent Verification](https://mlsanity.com/paper/2606.18861/) (2026)
- [Fine-grained Human Motion Understanding with Language Models](https://mlsanity.com/paper/2606.20888/) (2026)
- [Human-Centric Intelligence in the Era of Foundation Models: A Survey](https://mlsanity.com/paper/2608.18184/) (2026)

## Abstract (arXiv)

Advancements in Multimodal Large Language Models (MLLMs) have improved human motion understanding. However, these models remain constrained by their "instruct-only" nature, lacking interactivity and adaptability for diverse analytical perspectives. To address these challenges, we introduce ChatMotion, a multimodal multi-agent framework for human motion analysis. ChatMotion dynamically interprets user intent, decomposes complex tasks into meta-tasks, and activates specialized function modules for motion comprehension. It integrates multiple specialized modules, such as the MotionCore, to analyze human motion from various perspectives. Extensive experiments demonstrate ChatMotion's precision, adaptability, and user engagement for human motion understanding.