# ProEvent: An Event-centric Benchmark for Proactive Agents

arXiv: https://arxiv.org/abs/2607.17701 · HTML: https://mlsanity.com/paper/2607.17701/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 18 of its 28 references that are in the map.
- First posted on arXiv 20 Jul 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 18 of 18)

- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018)
- [Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset](https://mlsanity.com/paper/1909.05855/) (2019)
- [TravelPlanner: A Benchmark for Real-World Planning with Language Agents](https://mlsanity.com/paper/2402.01622/) (2024)
- [The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems](https://mlsanity.com/paper/1506.08909/) (2015)
- [Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaboration](https://mlsanity.com/paper/2305.13626/) (2023)
- [WebPilot: A Versatile and Autonomous Multi-Agent System for Web Task Execution with Strategic Exploration](https://mlsanity.com/paper/2408.15978/) (2024)
- [MultiWOZ 2.4: A Multi-Domain Task-Oriented Dialogue Dataset with Essential Annotation Corrections to Improve State Tracking Evaluation](https://mlsanity.com/paper/2104.00773/) (2021)
- [Recent Advances and Challenges in Task-oriented Dialog System](https://mlsanity.com/paper/2003.07490/) (2020)
- [ContextAgent: Context-Aware Proactive LLM Agents with Open-World Sensory Perceptions](https://mlsanity.com/paper/2505.14668/) (2025)
- [Interactive Agents: Simulating Counselor-Client Psychological Counseling via Role-Playing LLM-to-LLM Interactions](https://mlsanity.com/paper/2408.15787/) (2024)
- [RiSAWOZ: A Large-Scale Multi-Domain Wizard-of-Oz Dataset with Rich Semantic Annotations for Task-Oriented Dialogue Modeling](https://mlsanity.com/paper/2010.08738/) (2020)
- [FingerTip 20K: A Benchmark for Proactive and Personalized Mobile LLM Agents](https://mlsanity.com/paper/2507.21071/) (2025)
- [Large Language Models Perform Diagnostic Reasoning](https://mlsanity.com/paper/2307.08922/) (2023)
- [ProactiveVA: Proactive Visual Analytics with LLM-Based UI Agent](https://mlsanity.com/paper/2507.18165/) (2025)
- [Switch-based Active Deep Dyna-Q: Efficient Adaptive Planning for Task-Completion Dialogue Policy Learning](https://mlsanity.com/paper/1811.07550/) (2018)
- [Multi-Party Conversational Agents: A Survey](https://mlsanity.com/paper/2505.18845/) (2025)
- [Towards Effective MLLM Jailbreaking Through Balanced On-Topicness and OOD-Intensity](https://mlsanity.com/paper/2508.09218/) (2025)

## Built on by (most cited first, 2 of 2)

- [Agents in the Large: Perception-Centered Architecture for Persistent Agents](https://mlsanity.com/paper/2608.30478/) (2026)
- [Fresh at Decision, Stale at Action: Routing New Information to LLM Decisions Already Made](https://mlsanity.com/paper/2610.18126/) (2026)

## Abstract (arXiv)

Proactive agents are expected to anticipate user needs and provide autonomous assistance by perceiving environmental context without explicit instructions. A fundamental capability of such agents is to identify and track users' upcoming events, enabling continuous and event-specific assistance. For example, by recording the time and location of a planned hike, an agent can deliver weather reminders in advance or provide navigation support before departure. However, existing works on proactive agents largely overlook event-centric assistance, and the open-ended nature of proactive assistance poses challenges for reliable evaluation. To bridge these gaps, we introduce ProEvent, the first event-centric benchmark designed to assess an agent's ability to proactively maintain a user's timetable based on ongoing instant messaging chats. ProEvent provides synthesized yet realistic chats that consider the dynamic interaction among users, concurrent chat threads, and noise in the real world, and evaluates proactive agents on response timing, single-step response correctness, and multi-step response correctness. Experiments on eight LLMs and pipelines reveal that current agents frequently overact and struggle with event cancellation. Notably, even GPT-5.1 only reacts correctly in 26.7% of scenarios. Further qualitative analysis reveals fundamental limitations of current LLMs as proactive agents, particularly in detecting implicit events and reasoning from the user's first-person perspective.