# AGIR: Assessing 3D Gait Impairment with Reasoning based on LLMs

arXiv: https://arxiv.org/abs/2503.18141 · HTML: https://mlsanity.com/paper/2503.18141/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning.
- Builds on 19 of its 26 references that are in the map.
- First posted on arXiv 23 Mar 2025.

## Placed in

- LLM reasoning / [Clinical and biomedical applications](https://mlsanity.com/lr/clinical-and-biomedical-applications/)

## Builds on (most cited first, 19 of 19)

- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [MedCLIP: Contrastive Learning from Unpaired Medical Images and Text](https://mlsanity.com/paper/2210.10163/) (2022)
- [MotionGPT: Human Motion as a Foreign Language](https://mlsanity.com/paper/2306.14795/) (2023)
- [TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Texts](https://mlsanity.com/paper/2207.01696/) (2022)
- [Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive](https://mlsanity.com/paper/2402.13228/) (2024)
- [Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning](https://mlsanity.com/paper/2402.13950/) (2024)
- [PoseFormerV2: Exploring Frequency Domain for Efficient and Robust 3D Human Pose Estimation](https://mlsanity.com/paper/2303.17472/) (2023)
- [MotionAGFormer: Enhancing 3D Human Pose Estimation with a Transformer-GCNFormer Network](https://mlsanity.com/paper/2310.16288/) (2023)
- [LLMs are Good Action Recognizers](https://mlsanity.com/paper/2404.00532/) (2024)
- [PitVQA: Image-grounded Text Embedding LLM for Visual Question Answering in Pituitary Surgery](https://mlsanity.com/paper/2405.13949/) (2024)
- [Vision-based Estimation of MDS-UPDRS Gait Scores for Assessing Parkinson's Disease Motor Severity](https://mlsanity.com/paper/2007.08920/) (2020)
- [Estimating Parkinsonism Severity in Natural Gait Videos of Older Adults with Dementia](https://mlsanity.com/paper/2105.03464/) (2021)
- [Benchmarking Skeleton-based Motion Encoder Models for Clinical Applications: Estimating Parkinson's Disease Severity in Walking Sequences](https://mlsanity.com/paper/2405.17817/) (2024)
- [Insight: A Multi-Modal Diagnostic Pipeline using LLMs for Ocular Surface Disease Diagnosis](https://mlsanity.com/paper/2410.00292/) (2024)
- [Enhancing Gait Video Analysis in Neurodegenerative Diseases by Knowledge Augmentation in Vision Language Model](https://mlsanity.com/paper/2403.13756/) (2024)

## Built on by (most cited first, 3 of 3)

- [GAITGen: Disentangled Motion-Pathology Impaired Gait Generative Model -- Bringing Motion Generation to the Clinical Domain](https://mlsanity.com/paper/2503.22397/) (2025)
- [BiomechGPT: Extending Motion-Language Models to Clinical Motion Understanding](https://mlsanity.com/paper/2505.18465/) (2025)
- [DrGait: Biomechanically Grounded Visual Reasoning for Interpretable Clinical Gait Analysis](https://mlsanity.com/paper/2609.28796/) (2026)

## Abstract (arXiv)

Assessing gait impairment plays an important role in early diagnosis, disease monitoring, and treatment evaluation for neurodegenerative diseases. Despite its widespread use in clinical practice, it is limited by subjectivity and a lack of precision. While recent deep learning-based approaches have consistently improved classification accuracies, they often lack interpretability, hindering their utility in clinical decision-making. To overcome these challenges, we introduce AGIR, a novel pipeline consisting of a pre-trained VQ-VAE motion tokenizer and a subsequent Large Language Model (LLM) fine-tuned over pairs of motion tokens and Chain-of-Thought (CoT) reasonings. To fine-tune an LLM for pathological gait analysis, we first introduce a multimodal dataset by adding rationales dedicated to MDS-UPDRS gait score assessment to an existing PD gait dataset. We then introduce a two-stage supervised fine-tuning (SFT) strategy to enhance the LLM's motion comprehension with pathology-specific knowledge. This strategy includes: 1) a generative stage that aligns gait motions with analytic descriptions through bidirectional motion-description generation, 2) a reasoning stage that integrates logical Chain-of-Thought (CoT) reasoning for impairment assessment with UPDRS gait score. Validation on an existing dataset and comparisons with state-of-the-art methods confirm the robustness and accuracy of our pipeline, demonstrating its ability to assign gait impairment scores from motion input with clinically meaningful rationales.