# Real-Time Reasoning Agents in Evolving Environments

arXiv: https://arxiv.org/abs/2511.04898 · HTML: https://mlsanity.com/paper/2511.04898/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 24 of its 29 references that are in the map.
- First posted on arXiv 7 Nov 2025.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 24 of 24)

- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022)
- [Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs](https://mlsanity.com/paper/2412.21187/) (2024)
- [L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning](https://mlsanity.com/paper/2503.04697/) (2025)
- [Towards Real-Time Multi-Object Tracking](https://mlsanity.com/paper/1909.12605/) (2019)
- [ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving](https://mlsanity.com/paper/2309.17452/) (2023)
- [Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents](https://mlsanity.com/paper/2408.07199/) (2024)
- [From LLM Reasoning to Autonomous AI Agents: A Comprehensive Review](https://mlsanity.com/paper/2504.19678/) (2025)
- [OpenHelix: A Short Survey, Empirical Analysis, and Open-Source Dual-System VLA Model for Robotic Manipulation](https://mlsanity.com/paper/2505.03912/) (2025)
- [Benchmarking Reinforcement Learning Algorithms on Real-World Robots](https://mlsanity.com/paper/1809.07731/) (2018)
- [LLM-Powered Hierarchical Language Agent for Real-time Human-AI Coordination](https://mlsanity.com/paper/2312.15224/) (2023)
- [Agents Thinking Fast and Slow: A Talker-Reasoner Architecture](https://mlsanity.com/paper/2410.08328/) (2024)
- [THOUGHTTERMINATOR: Benchmarking, Calibrating, and Mitigating Overthinking in Reasoning Models](https://mlsanity.com/paper/2504.13367/) (2025)
- [Delay-Aware Model-Based Reinforcement Learning for Continuous Control](https://mlsanity.com/paper/2005.05440/) (2020)
- [HAZARD Challenge: Embodied Decision Making in Dynamically Changing Environments](https://mlsanity.com/paper/2401.12975/) (2024)
- [Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMs](https://mlsanity.com/paper/2507.02076/) (2025)
- [Detect or Track: Towards Cost-Effective Video Object Detection/Tracking](https://mlsanity.com/paper/1811.05340/) (2018)
- [Asynchronous Tool Usage for Real-Time Agents](https://mlsanity.com/paper/2410.21620/) (2024)
- [Reactive Reinforcement Learning in Asynchronous Environments](https://mlsanity.com/paper/1802.06139/) (2018)
- [Leveraging Dual Process Theory in Language Agent Framework for Real-time Simultaneous Human-AI Collaboration](https://mlsanity.com/paper/2502.11882/) (2025)
- [Enabling Realtime Reinforcement Learning at Scale with Staggered Asynchronous Inference](https://mlsanity.com/paper/2412.14355/) (2024)
- [Reflex First, Reflect Later: Latency-Aware Embodied LLM Agents for Dynamic Response](https://mlsanity.com/paper/2506.07223/) (2025)

## Built on by (most cited first, 15 of 15)

- [Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence](https://mlsanity.com/paper/2604.18292/) (2026)
- [GUI Agents for Continual Game Generation](https://mlsanity.com/paper/2605.28258/) (2026)
- [GameVerse: Can Vision-Language Models Learn from Video-based Reflection?](https://mlsanity.com/paper/2603.06656/) (2026)
- [Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application](https://mlsanity.com/paper/2606.12191/) (2026)
- [STT-Arena: A More Realistic Environment for Tool-Using with Spatio-Temporal Dynamics](https://mlsanity.com/paper/2605.18548/) (2026)
- [GLOVE: Global Verifier for LLM Memory-Environment Realignment](https://mlsanity.com/paper/2601.19249/) (2026)
- [Towards Generalist Game Players: An Investigation of Foundation Models in the Game Multiverse](https://mlsanity.com/paper/2605.09965/) (2026)
- [Engagement Process: Rethinking the Temporal Interface of Action and Observation](https://mlsanity.com/paper/2605.11484/) (2026)
- [OmniHarness: Harnessing Generalizable Visual Generation via Symbolic Policy Learning](https://mlsanity.com/paper/2609.16057/) (2026)
- [Never Stop Thinking: Continuous-Time Language Agents](https://mlsanity.com/paper/2609.17416/) (2026)
- [System Switch: When Should a Fast Decision Model Stop and Think?](https://mlsanity.com/paper/2610.09683/) (2026)
- [WMLLM: Self-Evolving Optimization Agents via Predict-Then-Act World Modeling](https://mlsanity.com/paper/2609.01608/) (2026)
- [ReLiveGym: Evaluating Long-Lived Agents over Weeks of Replayed Reality](https://mlsanity.com/paper/2610.00710/) (2026)
- [Separating Decision Time from Decision Quality in the Real-Time Gap of Distilled Deciders: Evidence from a Game and a Conveyor Simulator](https://mlsanity.com/paper/2610.04810/) (2026)
- [AgentTime: Can Agents Estimate and Control Their Own Runtime?](https://mlsanity.com/paper/2610.09944/) (2026)

## Abstract (arXiv)

Agents in the real world must make not only logical but also timely judgments. This requires continuous awareness of the dynamic environment: hazards emerge, opportunities arise, and other agents act, while the agent's reasoning is still unfolding. Despite advances in language model reasoning, existing approaches fail to account for this dynamic nature. We introduce real-time reasoning as a new problem formulation for agents in evolving environments and build Real-Time Reasoning Gym to demonstrate it. We study two paradigms for deploying language models in agents: (1) reactive agents, which employ language models with bounded reasoning computation for rapid responses, and (2) planning agents, which allow extended reasoning computation for complex problems. Our experiments show that even state-of-the-art models struggle with making logical and timely judgments in either paradigm. To address this limitation, we propose AgileThinker, which simultaneously engages both reasoning paradigms. AgileThinker consistently outperforms agents engaging only one reasoning paradigm as the task difficulty and time pressure rise, effectively balancing reasoning depth and response latency. Our work establishes real-time reasoning as a critical testbed for developing practical agents and provides a foundation for research in temporally constrained AI systems, highlighting a path toward real-time capable agents.