mlsanityData to 14 Oct 2026

Optimising Call Centre Operations using Reinforcement Learning: Value Iteration versus Proximal Policy Optimisation

arXiv 2507.18398 ↗First posted 24 Jul 2025cs.AIcited by 0

On the map: Deep reinforcement learning methods → RL for energy, networks and operations

Abstract (arXiv)

This paper investigates the application of Reinforcement Learning (RL) to optimise call routing in call centres to minimise client waiting time and staff idle time. Two methods are compared: a model-based approach using Value Iteration (VI) under known system dynamics, and a model-free approach using Proximal Policy Optimisation (PPO) that learns from experience. For the model-based approach, a theoretical model is used, while a simulation model combining Discrete Event Simulation (DES) with the OpenAI Gym environment is developed for model-free learning. Both models frame the problem as a Markov Decision Process (MDP) within a Skills-Based Routing (SBR) framework, with Poisson client arrivals and exponentially distributed service and abandonment times. For policy evaluation, random, VI, and PPO policies are evaluated using the simulation model. After 1,000 test episodes, PPO consistently achives the highest rewards, along with the lowest client waiting time and staff idle time, despite requiring longer training time.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 5 of its 13 references that are in the map. First posted on arXiv 24 Jul 2025.

Builds on

5 of 13 references.

References · 5

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2016OpenAI Gymcited by 1,736
  3. 2020Queueing Network Controls via Deep Reinforcement Learningcited by 41
  4. 2020RL-QN: A Reinforcement Learning Framework for Optimal Control of Queueing Systemscited by 11
  5. 2017MDP environments for the OpenAI Gymcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Empirical Evaluation of Policy-Based Reinforcement Learning for Dynamic Service Control in an M/M/1 Queue
  2. 2025Demonstration of effective UCB-based routing in skill-based queues on real-world data
  3. 2025Simulation-Driven Reinforcement Learning in Queuing Network Routing Optimization
  4. 2024Reinforcement Learning for Intensity Control: An Application to Choice-Based Network Revenue Management
  5. 2024Efficient Reinforcement Learning for Routing Jobs in Heterogeneous Queueing Systems
  6. 2026Reinforcement Learning in Operational Research: A Technical Review and Practical Roadmap
  7. 2024Learning State-Dependent Policy Parametrizations for Dynamic Technician Routing with Rework
  8. 2026Mathematical methods of reinforcement learning