# Separating Decision Time from Decision Quality in the Real-Time Gap of Distilled Deciders: Evidence from a Game and a Conveyor Simulator

arXiv: https://arxiv.org/abs/2610.04810 · HTML: https://mlsanity.com/paper/2610.04810/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 58 references that are in the map.
- First posted on arXiv 3 Oct 2026.

## Builds on (most cited first, 30 of 31)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Challenges of Real-World Reinforcement Learning](https://mlsanity.com/paper/1904.12901/) (2019)
- [Distilling System 2 into System 1](https://mlsanity.com/paper/2407.06023/) (2024)
- [A Survey on Large Language Model-Based Game Agents](https://mlsanity.com/paper/2404.02039/) (2024)
- [How Many Random Seeds? Statistical Power Analysis in Deep Reinforcement Learning Experiments](https://mlsanity.com/paper/1806.08295/) (2018)
- [VideoGameBench: Can Vision-Language Models complete popular video games?](https://mlsanity.com/paper/2505.18134/) (2025)
- [Orak: A Foundational Benchmark for Training and Evaluating LLM Agents on Diverse Video Games](https://mlsanity.com/paper/2506.03610/) (2025)
- [GameWorld: Towards Standardized and Verifiable Evaluation of Multimodal Game Agents](https://mlsanity.com/paper/2604.07429/) (2026)
- [At Human Speed: Deep Reinforcement Learning with Action Delay](https://mlsanity.com/paper/1810.07286/) (2018)
- [Can VLMs Play Action Role-Playing Games? Take Black Myth Wukong as a Study Case](https://mlsanity.com/paper/2409.12889/) (2024)
- [SwarmBrain: Embodied agent for real-time strategy game StarCraft II via large language models](https://mlsanity.com/paper/2401.17749/) (2024)
- [An Analysis of Frame-skipping in Reinforcement Learning](https://mlsanity.com/paper/2102.03718/) (2021)
- [Real-Time Reasoning Agents in Evolving Environments](https://mlsanity.com/paper/2511.04898/) (2025)
- [Pre-registration for Predictive Modeling](https://mlsanity.com/paper/2311.18807/) (2023)
- [Let's Play Again: Variability of Deep Reinforcement Learning Agents in Atari Environments](https://mlsanity.com/paper/1904.06312/) (2019)
- [TIC-VLA: A Think-in-Control Vision-Language-Action Model for Robot Navigation in Dynamic Environments](https://mlsanity.com/paper/2602.02459/) (2026)
- [GVGAI-LLM: Evaluating Large Language Model Agents with Infinite Games](https://mlsanity.com/paper/2508.08501/) (2025)
- [Language is Power: Representing States Using Natural Language in Reinforcement Learning](https://mlsanity.com/paper/1910.02789/) (2019)
- [World Action Models in Real Time: An Empirical Study of Smooth Execution via Asynchronous Deployment](https://mlsanity.com/paper/2608.01880/) (2026)
- [Dynamic Frame skip Deep Q Network](https://mlsanity.com/paper/1605.05365/) (2016)
- [Reflex First, Reflect Later: Latency-Aware Embodied LLM Agents for Dynamic Response](https://mlsanity.com/paper/2506.07223/) (2025)
- [Bridging Large-Model Reasoning and Real-Time Control via Agentic Fast-Slow Planning](https://mlsanity.com/paper/2604.01681/) (2026)
- [Model-Based Reinforcement Learning under Random Observation Delays](https://mlsanity.com/paper/2509.20869/) (2025)
- [Accelerating Multi-modal LLM Gaming Performance via Input Prediction and Mishit Correction](https://mlsanity.com/paper/2512.17250/) (2025)
- [The Latent Bridge: A Continuous Slow-Fast Channel for Real-Time Game Agents](https://mlsanity.com/paper/2606.24470/) (2026)
- [Playing DOOM with 1.3M Parameters: Specialized Small Models vs Large Language Models for Real-Time Game Control](https://mlsanity.com/paper/2604.07385/) (2026)
- [Latency-Tolerant Cloud-Edge Collaborative Vision-Language-Action Models via Emergent Representational Specialization](https://mlsanity.com/paper/2608.00569/) (2026)
- [Learning to Act While Waiting: RL Finetuning of Generalist Robot Policies Under Inference Latency](https://mlsanity.com/paper/2608.23831/) (2026)
- [Reinforcement Learning for Real-Time Vision-Language-Action Policies](https://mlsanity.com/paper/2609.18207/) (2026)
- [For How Long Should We Be Punching? Learning Action Duration in Fighting Games](https://mlsanity.com/paper/2605.20911/) (2026)

## Abstract (arXiv)

Real-time agents are often evaluated with the world paused, or with decision time rounded to whole ticks (tick conversion). Tick conversion predicts almost no loss for a learned decider whose hit accuracy falls by 14.8 percentage points (pp) on the same games in asynchronous play. We instead split the decider's gap to a zero-latency teacher into time and quality components. In ViZDoom, we train a small decider by imitating a scripted teacher, run the game on wall-clock time, and add a control in which the teacher waits for a call to the decider's server before deciding. On a Windows host, three preregistered studies put the time component at 6.5-8.7 pp and the quality component at 6.3-8.6 pp. On a Linux host that skips 0.08-0.09% of ticks, the time component disappeared (-0.1 pp; paired 95% bootstrap interval [-0.4, 0.0]) while the quality component remained (5.6 pp [3.7, 7.5]). Retraining on teacher-labelled states from the decider's own play (DAgger) improved it on new games on both hosts (2.6 pp [0.4, 4.9] and 3.1 pp [1.1, 5.2]), a preregistered partial success. Imposing one delay schedule on every arm in live play left a quality component of 5.8 pp [4.0, 7.5] on 100 new games, and retraining cut the decider's disagreement with the teacher on its own states from 22.7% to 14.9%. In a conveyor simulator with a 400 ms deadline, delay past it erased the quality component. A latency-matched control whose delays match the decider's shows which remedy to try.