# Engagement Process: Rethinking the Temporal Interface of Action and Observation

arXiv: https://arxiv.org/abs/2605.11484 · HTML: https://mlsanity.com/paper/2605.11484/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 25 of its 42 references that are in the map.
- First posted on arXiv 12 May 2026.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)

## Builds on (most cited first, 25 of 25)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [Large Language Monkeys: Scaling Inference Compute with Repeated Sampling](https://mlsanity.com/paper/2407.21787/) (2024)
- [OpenThoughts: Data Recipes for Reasoning Models](https://mlsanity.com/paper/2506.04178/) (2025)
- [π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilities](https://mlsanity.com/paper/2604.15483/) (2026)
- [Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities](https://mlsanity.com/paper/2503.04721/) (2025)
- [Revisiting Active Perception](https://mlsanity.com/paper/1603.02729/) (2016)
- [A Survey on Active Simultaneous Localization and Mapping: State of the Art and New Frontiers](https://mlsanity.com/paper/2207.00254/) (2022)
- [Language Model Can Listen While Speaking](https://mlsanity.com/paper/2408.02622/) (2024)
- [A Full-duplex Speech Dialogue Scheme Based On Large Language Models](https://mlsanity.com/paper/2405.19487/) (2024)
- [Robotouille: An Asynchronous Planning Benchmark for LLM Agents](https://mlsanity.com/paper/2502.05227/) (2025)
- [Real-Time Reasoning Agents in Evolving Environments](https://mlsanity.com/paper/2511.04898/) (2025)
- [Asynchronous Tool Usage for Real-Time Agents](https://mlsanity.com/paper/2410.21620/) (2024)
- [Handling Delay in Real-Time Reinforcement Learning](https://mlsanity.com/paper/2503.23478/) (2025)
- [AViLA: Asynchronous Vision-Language Agent for Streaming Multimodal Data Interaction](https://mlsanity.com/paper/2506.18472/) (2025)
- [Reflex First, Reflect Later: Latency-Aware Embodied LLM Agents for Dynamic Response](https://mlsanity.com/paper/2506.07223/) (2025)
- [POMDPs in Continuous Time and Discrete Spaces](https://mlsanity.com/paper/2010.01014/) (2020)
- [From Static Inference to Dynamic Interaction: A Survey of Streaming Large Language Models](https://mlsanity.com/paper/2603.04592/) (2026)

## Built on by (most cited first, 2 of 2)

- [Athena-Brain Technical Report: An Efficient Robot Brain for General Intelligence and Embodied Interaction](https://mlsanity.com/paper/2607.18985/) (2026)
- [When Is an Agent Evaluation Over? Outcome Finality and Cross-Unit Separation](https://mlsanity.com/paper/2608.14940/) (2026)

## Abstract (arXiv)

Task completion in digital and physical environments increasingly involves complex temporal interaction, where actions and observations unfold over different time scales rather than align with fixed observation--action steps. To model such interactions, we propose Engagement Process (EP), an interaction formalism that inherits the decision-theoretic structure of POMDPs while making time explicit in the action--observation interface. EP represents actions and observations as decoupled event streams along time, rather than updates paired at fixed decision steps. This interface captures single-agent timing issues such as deliberation latency, delayed feedback, and persistent actions, while supporting richer agent-side organization, multi-rate coordination, and compositional interaction among subsystems. Across toy, LLM-agent, and learning experiments, EP exposes temporal behaviors hidden by step-based interfaces and enables policies to adapt under explicit time costs.