# Playing DOOM with 1.3M Parameters: Specialized Small Models vs Large Language Models for Real-Time Game Control

arXiv: https://arxiv.org/abs/2604.07385 · HTML: https://mlsanity.com/paper/2604.07385/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 25 references that are in the map.
- First posted on arXiv 8 Apr 2026.

## Builds on (most cited first, 19 of 19)

- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference](https://mlsanity.com/paper/2412.13663/) (2024)
- [Diffusion Models Are Real-Time Game Engines](https://mlsanity.com/paper/2408.14837/) (2024)
- [Small Language Models are the Future of Agentic AI](https://mlsanity.com/paper/2506.02153/) (2025)
- [Compressing Neural Networks with the Hashing Trick](https://mlsanity.com/paper/1504.04788/) (2015)
- [MLPerf Tiny Benchmark](https://mlsanity.com/paper/2106.07597/) (2021)
- [Playing FPS Games with Deep Reinforcement Learning](https://mlsanity.com/paper/1609.05521/) (2016)
- [Large Language Models and Games: A Survey and Roadmap](https://mlsanity.com/paper/2402.18659/) (2024)
- [Hash Embeddings for Efficient Word Representations](https://mlsanity.com/paper/1709.03933/) (2017)
- [Learning to Act by Predicting the Future](https://mlsanity.com/paper/1611.01779/) (2016)
- [Can Large Language Models Play Text Games Well? Current State-of-the-Art and Open Questions](https://mlsanity.com/paper/2304.02868/) (2023)
- [PyLate: Flexible Training and Retrieval for Late Interaction Models](https://mlsanity.com/paper/2508.03555/) (2025)
- [Task-Specific Efficiency Analysis: When Small Language Models Outperform Large Language Models](https://mlsanity.com/paper/2603.21389/) (2026)
- [Atari-GPT: Benchmarking Multimodal Large Language Models as Low-Level Policies in Atari Games](https://mlsanity.com/paper/2408.15950/) (2024)
- [ASCIIBench: Evaluating Language-Model-Based Understanding of Visually-Oriented Text](https://mlsanity.com/paper/2512.04125/) (2025)

## Built on by (most cited first, 2 of 2)

- [System Switch: When Should a Fast Decision Model Stop and Think?](https://mlsanity.com/paper/2610.09683/) (2026)
- [Separating Decision Time from Decision Quality in the Real-Time Gap of Distilled Deciders: Evidence from a Game and a Conveyor Simulator](https://mlsanity.com/paper/2610.04810/) (2026)

## Abstract (arXiv)

We present SauerkrautLM-Doom-MultiVec, a 1.3 million parameter model that plays the classic first-person shooter DOOM in real time, outperforming large language models up to 92,000x its size, including Nemotron-120B, Qwen3.5-27B, and GPT-4o-mini. Our model combines a ModernBERT encoder with hash embeddings, depth-aware token representations, and an attention pooling classification head to select game actions from ASCII frame representations at 31ms per decision. Trained on just 31,000 human gameplay demonstrations, it achieves 178 frags in 10 episodes (17.8 per episode) in the defend_the_center scenario, more than all tested LLMs combined (13 frags total). All agents receive equivalent input: ASCII frames and depth maps. Despite having 92,000x fewer parameters than Nemotron-120B, our model is the only agent that actively engages enemies rather than purely evading them. These results demonstrate that small, task-specific models trained on domain-appropriate data can decisively outperform general-purpose LLMs at real-time control tasks, at a fraction of the inference cost, with deployment capability on consumer hardware.