# Can Large Models Fool the Eye? A New Turing Test for Biological Animation

arXiv: https://arxiv.org/abs/2508.06072 · HTML: https://mlsanity.com/paper/2508.06072/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 47 references that are in the map.
- First posted on arXiv 8 Aug 2025.

## Builds on (most cited first, 26 of 26)

- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [MMBench: Is Your Multi-modal Model an All-around Player?](https://mlsanity.com/paper/2307.06281/) (2023)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://mlsanity.com/paper/2412.05271/) (2024)
- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://mlsanity.com/paper/2403.04132/) (2024)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools](https://mlsanity.com/paper/2406.12793/) (2024)
- [BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset](https://mlsanity.com/paper/2307.04657/) (2023)
- [OpenAssistant Conversations -- Democratizing Large Language Model Alignment](https://mlsanity.com/paper/2304.07327/) (2023)
- [PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change](https://mlsanity.com/paper/2206.10498/) (2022)
- [Human Preference Score: Better Aligning Text-to-Image Models with Human Preference](https://mlsanity.com/paper/2303.14420/) (2023)
- [WildVision: Evaluating Vision-Language Models in the Wild with Human Preferences](https://mlsanity.com/paper/2406.11069/) (2024)
- [GameArena: Evaluating LLM Reasoning through Live Computer Games](https://mlsanity.com/paper/2412.06394/) (2024)
- [GAIA: Rethinking Action Quality Assessment for AI-Generated Videos](https://mlsanity.com/paper/2406.06087/) (2024)
- [Automated Multi-level Preference for MLLMs](https://mlsanity.com/paper/2405.11165/) (2024)
- [Dynamic Multimodal Evaluation with Flexible Complexity by Vision-Language Bootstrapping](https://mlsanity.com/paper/2410.08695/) (2024)
- [OBI-Bench: Can LMMs Aid in Study of Ancient Script on Oracle Bones?](https://mlsanity.com/paper/2412.01175/) (2024)
- [PuzzleBench: A Fully Dynamic Evaluation Framework for Large Multimodal Models on Puzzle Solving](https://mlsanity.com/paper/2504.10885/) (2025)

## Built on by (most cited first, 6 of 6)

- [MACEval: A Multi-Agent Continual Evaluation Network for Large Models](https://mlsanity.com/paper/2511.09139/) (2025)
- [Evaluating from Benign to Dynamic Adversarial: A Squid Game for Large Language Models](https://mlsanity.com/paper/2511.10691/) (2025)
- [ROFI: A Deep Learning-Based Ophthalmic Sign-Preserving and Reversible Patient Face Anonymizer](https://mlsanity.com/paper/2510.11073/) (2025)
- [PriceSeer: Evaluating Large Language Models in Real-Time Stock Prediction](https://mlsanity.com/paper/2601.06088/) (2025)
- [Evaluating point-light biological motion in multimodal large language models](https://mlsanity.com/paper/2509.23517/) (2025)
- [Invisible in Space, Visible in Time: Motion Vision CAPTCHA against GUI Agents](https://mlsanity.com/paper/2609.27461/) (2026)

## Abstract (arXiv)

Evaluating the abilities of large models and manifesting their gaps are challenging. Current benchmarks adopt either ground-truth-based score-form evaluation on static datasets or indistinct textual chatbot-style human preferences collection, which may not provide users with immediate, intuitive, and perceptible feedback on performance differences. In this paper, we introduce BioMotion Arena, a novel framework for evaluating large language models (LLMs) and multimodal large language models (MLLMs) via visual animation. Our methodology draws inspiration from the inherent visual perception of motion patterns characteristic of living organisms that utilizes point-light source imaging to amplify the performance discrepancies between models. Specifically, we employ a pairwise comparison evaluation and collect more than 45k votes for 53 mainstream LLMs and MLLMs on 90 biological motion variants. Data analyses show that the crowd-sourced human votes are in good agreement with those of expert raters, demonstrating the superiority of our BioMotion Arena in offering discriminative feedback. We also find that over 90% of evaluated models, including the cutting-edge open-source InternVL3 and proprietary Claude-4 series, fail to produce fundamental humanoid point-light groups, much less smooth and biologically plausible motions. This enables BioMotion Arena to serve as a challenging benchmark for performance visualization and a flexible evaluation framework without restrictions on ground-truth.