mlsanityData to 14 Oct 2026

Switch-JustDance: Benchmarking Whole Body Motion Tracking Controllers Using a Commercial Console Game

arXiv 2511.17925 ↗First posted 22 Nov 2025cs.RO cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent advances in whole-body robot control have enabled humanoid and legged robots to perform increasingly agile and coordinated motions. However, standardized benchmarks for evaluating these capabilities in real-world settings, and in direct comparison to humans, remain scarce. Existing evaluations often rely on pre-collected human motion datasets or simulation-based experiments, which limit reproducibility, overlook hardware factors, and hinder fair human-robot comparisons. We present Switch-JustDance, a low-cost and reproducible benchmarking pipeline that leverages motion-sensing console games, Just Dance on the Nintendo Switch, to evaluate robot whole-body control. Using Just Dance on the Nintendo Switch as a representative platform, Switch-JustDance converts in-game choreography into robot-executable motions through streaming, motion reconstruction, and motion retargeting modules and enables users to evaluate controller performance through the game's built-in scoring system. We first validate the evaluation properties of Just Dance, analyzing its reliability, validity, sensitivity, and potential sources of bias. Our results show that the platform provides consistent and interpretable performance measures, making it a suitable tool for benchmarking embodied AI. Building on this foundation, we benchmark three state-of-the-art humanoid whole-body controllers on hardware and provide insights into their relative strengths and limitations.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 41 references that are in the map. First posted on arXiv 22 Nov 2025.

Builds on

24 of 41 references.

References · 24

  1. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
  2. 2016Playing for Data: Ground Truth from Computer Gamescited by 818
  3. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 478
  4. 2017StarCraft II: A New Challenge for Reinforcement Learningcited by 274
  5. 2021Robust Motion In-betweeningcited by 229
  6. 2024OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learningcited by 189
  7. 2016ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learningcited by 179
  8. 2024Open-TeleVision: Teleoperation with Immersive Active Visual Feedbackcited by 145
  9. 2024World-Grounded Human Motion Recovery via Gravity-View Coordinatescited by 137
  10. 2024Learning Human-to-Humanoid Real-Time Whole-Body Teleoperationcited by 130
  11. 2024MaskedMimic: Unified Physics-Based Character Control Through Masked Motion Inpaintingcited by 116
  12. 2024HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulationcited by 112
12 more
  1. 2025TWIST: Teleoperated Whole-Body Imitation Systemcited by 101
  2. 2024HOVER: Versatile Neural Whole-Body Controller for Humanoid Robotscited by 91
  3. 2025GMT: General Motion Tracking for Humanoid Whole-Body Controlcited by 83
  4. 2022QuestSim: Human Motion Tracking from Sparse Sensors with Simulated Avatarscited by 77
  5. 2025KungfuBot: Physics-Based Humanoid Whole-Body Control for Learning Highly-Dynamic Skillscited by 52
  6. 2025AMO: Adaptive Motion Optimization for Hyper-Dexterous Humanoid Whole-Body Controlcited by 51
  7. 2025Track Any Motions under Any Disturbancescited by 35
  8. 2020Retrospective Analysis of the 2019 MineRL Competition on Sample Efficient Reinforcement Learningcited by 18
  9. 2025Unleashing Humanoid Reaching Potential via Real-world-Ready Skill Spacecited by 14
  10. 2024Tailoring Solution Accuracy for Fast Whole-body Model Predictive Control of Legged Robotscited by 13
  11. 2024Cafe-Mpc: A Cascaded-Fidelity Model Predictive Control Framework with Tuning-Free Whole-Body Controlcited by 10
  12. 2025Generating Physically Realistic and Directable Human Motions from Multi-Modal Inputscited by 6

Built on by

Not cited yet.