# ProActor: Timing-Aware Reinforcement Learning for Proactive Task Scheduling Agents

arXiv: https://arxiv.org/abs/2605.24900 · HTML: https://mlsanity.com/paper/2605.24900/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 29 of its 65 references that are in the map.
- First posted on arXiv 24 May 2026.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)

## Builds on (most cited first, 29 of 29)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [Hindsight Experience Replay](https://mlsanity.com/paper/1707.01495/) (2017)
- [IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures](https://mlsanity.com/paper/1802.01561/) (2018)
- [Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset](https://mlsanity.com/paper/1909.05855/) (2019)
- [Fine-Grained Human Feedback Gives Better Rewards for Language Model Training](https://mlsanity.com/paper/2306.01693/) (2023)
- [ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL](https://mlsanity.com/paper/2402.19446/) (2024)
- [CollabLLM: From Passive Responders to Active Collaborators](https://mlsanity.com/paper/2502.00640/) (2025)
- [Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaboration](https://mlsanity.com/paper/2305.13626/) (2023)
- [Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistance](https://mlsanity.com/paper/2410.12361/) (2024)
- [Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2104.00783/) (2021)
- [Towards Human-centered Proactive Conversational Agents](https://mlsanity.com/paper/2404.12670/) (2024)
- [Proactive Conversational Agents with Inner Thoughts](https://mlsanity.com/paper/2501.00383/) (2024)
- [Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Training](https://mlsanity.com/paper/2503.18929/) (2025)
- [Proactive Assistant Dialogue Generation from Streaming Egocentric Videos](https://mlsanity.com/paper/2506.05904/) (2025)
- [ComPeer: A Generative Conversational Agent for Proactive Peer Support](https://mlsanity.com/paper/2407.18064/) (2024)
- [QuaRL: Quantization for Fast and Environmentally Sustainable Reinforcement Learning](https://mlsanity.com/paper/1910.01055/) (2019)
- [The Sample-Complexity of General Reinforcement Learning](https://mlsanity.com/paper/1308.4828/) (2013)
- [PRINCIPLES: Synthetic Strategy Memory for Proactive Dialogue Agents](https://mlsanity.com/paper/2509.17459/) (2025)
- [DistFlow: A Fully Distributed RL Framework for Scalable and Efficient LLM Post-Training](https://mlsanity.com/paper/2507.13833/) (2025)
- [ProactiveEval: A Unified Evaluation Framework for Proactive Dialogue Agents](https://mlsanity.com/paper/2508.20973/) (2025)
- [LLAMAPIE: Proactive In-Ear Conversation Assistants](https://mlsanity.com/paper/2505.04066/) (2025)
- [On the Effectiveness of Offline RL for Dialogue Response Generation](https://mlsanity.com/paper/2307.12425/) (2023)
- [Arctic Long Sequence Training: Scalable And Efficient Training For Multi-Million Token Sequences](https://mlsanity.com/paper/2506.13996/) (2025)
- [Towards Outcome-Oriented, Task-Agnostic Evaluation of AI Agents](https://mlsanity.com/paper/2511.08242/) (2025)
- [Beyond Task-Oriented and Chitchat Dialogues: Proactive and Transition-Aware Conversational Agents](https://mlsanity.com/paper/2511.08835/) (2025)
- [Hierarchical Reinforcement Learning for Temporal Abstraction of Listwise Recommendation](https://mlsanity.com/paper/2409.07416/) (2024)

## Built on by (most cited first, 3 of 3)

- [Do Proactive Agents Really Need an LLM to Decide When to Wake and What to Anchor?](https://mlsanity.com/paper/2605.30152/) (2026)
- [Foundations of Proactive Agents: Principles, Technical Layers, and Proactivity-Gym](https://mlsanity.com/paper/2609.37267/) (2026)
- [StreamDecisionBench: Evaluating Decisions in Force on Evolving Language Streams](https://mlsanity.com/paper/2609.38612/) (2026)

## Abstract (arXiv)

Proactive task-oriented agents must autonomously anticipate user needs, identify actionable opportunities, and trigger software actions at appropriate moments - fundamentally shifting from reactive systems that await explicit instructions. However, existing approaches lack generalizable end-to-end solutions for measuring and optimizing such anticipatory behaviors. This paper introduces ProActor, a unified framework for conversational task scheduling that integrates: (1) a domain-agnostic automated annotation methodology that enables scalable proactiveness reinforcement learning (RL) by generating full opportunity time windows instead of rigid point labels, (2) systematic proactiveness metrics capturing both timing quality and reference action alignment, and (3) RL optimization using GRPO with various reward designs. Our insight is that RULER-based rewards with proactiveness rubrics are crucial for improving timing quality, and that proactiveness optimization enabled by stage-aware composite rewards is key to balancing timing quality and reference action alignment. Timing-aware RL requires extensive exploration, demanding efficient infrastructure. We develop ART-F, an adaptive framework combining request-adaptive inference clusters with DDP-based training on single-node multi-GPU systems, enabling LoRA training of 4-bit Qwen2.5-14B-ProActor-Q4 with 4-8x speedups. Experiments on two newly auto-annotated datasets demonstrate significant improvements in proactive timing while maintaining action consistency comparable to state-of-the-art (SOTA) baselines. Ablations validate the effectiveness of distinct composite reward variations.