mlsanityData to 14 Oct 2026

FormCoach: Lift Smarter, Not Harder

arXiv 2508.07501 ↗First posted 10 Aug 2025cs.CV cs.HCcited by 0

Not on the map. Why?

Abstract (arXiv)

Good form is the difference between strength and strain, yet for the fast-growing community of at-home fitness enthusiasts, expert feedback is often out of reach. FormCoach transforms a simple camera into an always-on, interactive AI training partner, capable of spotting subtle form errors and delivering tailored corrections in real time, leveraging vision-language models (VLMs). We showcase this capability through a web interface and benchmark state-of-the-art VLMs on a dataset of 1,700 expert-annotated user-reference video pairs spanning 22 strength and mobility exercises. To accelerate research in AI-driven coaching, we release both the dataset and an automated, rubric-based evaluation pipeline, enabling standardized comparison across models. Our benchmarks reveal substantial gaps compared to human-level coaching, underscoring both the challenges and opportunities in integrating nuanced, context-aware movement analysis into interactive AI systems. By framing form correction as a collaborative and creative process between humans and machines, FormCoach opens a new frontier in embodied AI.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 33 references that are in the map. First posted on arXiv 10 Aug 2025.

Builds on

13 of 33 references.

References · 13

  1. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  2. 2024Qwen2 Technical Reportcited by 2,120
  3. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,162
  4. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  5. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 518
  6. 2013DeepPose: Human Pose Estimation via Deep Neural Networkscited by 305
  7. 2019Gradio: Hassle-Free Sharing and Testing of ML Models in the Wildcited by 178
  8. 2023MotionGPT: Finetuned LLMs Are General-Purpose Motion Generatorscited by 145
  9. 2024MotionLLM: Understanding Human Behaviors from Human Motions and Videoscited by 90
  10. 2024MotionFix: Text-Driven 3D Human Motion Editingcited by 64
  11. 2020Pose Trainer: Correcting Exercise Posture using Pose Estimationcited by 18
  12. 2022PoseCoach: A Customizable Analysis and Visualization System for Video-based Running Coachingcited by 8
1 more
  1. 2025Can Vision Language Models Understand Mimed Actions?cited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026FitAQA: A Benchmark of Fitness Action Quality Assessment for Multimodal Large Language Models
  2. 2024What to Say and When to Say it: Live Fitness Coaching as a Testbed for Situated Interaction
  3. 2026BoT-Feedback: Grounding Multimodal Reasoning in Biomechanical Evidence for Explainable Human Action Feedback
  4. 2024ExpertAF: Expert Actionable Feedback from Video
  5. 2026Can Vision Language Models Judge Action Quality? An Empirical Evaluation
  6. 2026SherpaAI: A Multi-modal Solution for Delivering Personalized and Adaptive Fitness Interventions ↗
  7. 2026EmbodiedMidtrain: Bridging the Gap between Vision-Language Models and Vision-Language-Action Models via Mid-training
  8. 2025Igniting VLMs toward the Embodied Space