# Open-Ended Video Game Glitch Detection with Agentic Reasoning and Temporal Grounding

arXiv: https://arxiv.org/abs/2604.07818 · HTML: https://mlsanity.com/paper/2604.07818/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 21 of its 39 references that are in the map.
- First posted on arXiv 9 Apr 2026.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 21 of 21)

- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://mlsanity.com/paper/2412.05271/) (2024)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [Future Frame Prediction for Anomaly Detection -- A New Baseline](https://mlsanity.com/paper/1712.09867/) (2017)
- [VideoAgent: Long-form Video Understanding with Large Language Model as Agent](https://mlsanity.com/paper/2403.10517/) (2024)
- [Learning Temporal Regularity in Video Sequences](https://mlsanity.com/paper/1604.04574/) (2016)
- [Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervision](https://mlsanity.com/paper/2007.04687/) (2020)
- [A Survey on Large Language Model-Based Game Agents](https://mlsanity.com/paper/2404.02039/) (2024)
- [A Survey of Single-Scene Video Anomaly Detection](https://mlsanity.com/paper/2004.05993/) (2020)
- [VideoMind: A Chain-of-LoRA Agent for Temporal-Grounded Video Reasoning](https://mlsanity.com/paper/2503.13444/) (2025)
- [Augmenting Automated Game Testing with Deep Reinforcement Learning](https://mlsanity.com/paper/2103.15819/) (2021)
- [GlitchBench: Can large multimodal models detect video game glitches?](https://mlsanity.com/paper/2312.05291/) (2023)
- [Automated Video Game Testing Using Synthetic and Human-Like Agents](https://mlsanity.com/paper/1906.00317/) (2019)
- [PhysGame: Uncovering Physical Commonsense Violations in Gameplay Videos](https://mlsanity.com/paper/2412.01800/) (2024)
- [Dense Video Captioning: A Survey of Techniques, Datasets and Evaluation Protocols](https://mlsanity.com/paper/2311.02538/) (2023)
- [CLIP meets GamePhysics: Towards bug identification in gameplay videos using zero-shot transfer learning](https://mlsanity.com/paper/2203.11096/) (2022)
- [Large Language Models are Pretty Good Zero-Shot Video Game Bug Detectors](https://mlsanity.com/paper/2210.02506/) (2022)
- [VideoGameQA-Bench: Evaluating Vision-Language Models for Video Game Quality Assurance](https://mlsanity.com/paper/2505.15952/) (2025)
- [Inspector: Pixel-Based Automated Game Testing via Exploration, Detection, and Investigation](https://mlsanity.com/paper/2207.08379/) (2022)
- [Learning to Identify Perceptual Bugs in 3D Video Games](https://mlsanity.com/paper/2202.12884/) (2022)

## Built on by (most cited first, 3 of 3)

- [AI for Games in the Foundation Model Era](https://mlsanity.com/paper/2609.16679/) (2026)
- [RoboGaze: Evaluating Robot World Models via Structured Vision-Language Analysis](https://mlsanity.com/paper/2606.28385/) (2026)
- [WorldAuditBench: Interactive 3D World Auditing with Multimodal Agents](https://mlsanity.com/paper/2609.40325/) (2026)

## Abstract (arXiv)

Open-ended video game glitch detection aims to identify glitches in gameplay videos, describe them in natural language, and localize when they occur. Unlike conventional game glitch understanding tasks which have largely been framed as image-level recognition or closed-form question answering, this task requires reasoning about game-specific dynamics such as mechanics, physics, rendering, animation, and expected state transitions directly over continuous gameplay videos and distinguishing true glitches from unusual but valid in-game events. To support this task, we introduce VideoGlitchBench, the first benchmark for open-ended video game glitch detection with temporal localization. VideoGlitchBench contains 5,238 gameplay videos from 120 games, each annotated with detailed glitch descriptions and precise temporal spans, enabling unified evaluation of semantic understanding and temporal grounding. We further propose GliDe, an agentic framework with three key components: a game-aware contextual memory for informed reasoning, a debate-based reflector for multi-perspective glitch detection and verification, and an event-level grounding module that recovers complete glitch intervals from fragmented temporal evidence. We also design a task-specific evaluation protocol that jointly measures semantic fidelity and temporal accuracy. Experiments show that this task remains highly challenging for current multimodal models, while GliDe achieves substantially stronger performance than corresponding vanilla model baselines.