Improving After-sales Service: Deep Reinforcement Learning for Dynamic Time Slot Assignment with Commitments and Customer Preferences
On the map: Deep reinforcement learning methods → RL for energy, networks and operations
Abstract (arXiv)
Problem definition: For original equipment manufacturers (OEMs), high-tech maintenance is a strategic component in after-sales services, involving close coordination between customers and service engineers. Each customer suggests several time slots for their maintenance task, from which the OEM must select one. This decision needs to be made promptly to support customers' planning. At the end of each day, routes for service engineers are planned to fulfill the tasks scheduled for the following day. We study this hierarchical and sequential decision-making problem-the Dynamic Time Slot Assignment Problem with Commitments and Customer Preferences (DTSAP-CCP)-in this paper. Methodology/results: Two distinct approaches are proposed: 1) an attention-based deep reinforcement learning with rollout execution (ADRL-RE) and 2) a scenario-based planning approach (SBP). The ADRL-RE combines a well-trained attention-based neural network with a rollout framework for online trajectory simulation. To support the training, we develop a neural heuristic solver that provides rapid route planning solutions, enabling efficient learning in complex combinatorial settings. The SBP approach samples several scenarios to guide the time slot assignment. Numerical experiments demonstrate the superiority of ADRL-RE and the stability of SBP compared to both rule-based and rollout-based approaches. Furthermore, the strong practicality of ADRL-RE is verified in a case study of after-sales service for large medical equipment. Implications: This study provides OEMs with practical decision-support tools for dynamic maintenance scheduling, balancing customer preferences and operational efficiency. In particular, our ADRL-RE shows strong real-world potential, supporting timely and customer-aligned maintenance scheduling.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 7 of its 40 references that are in the map. First posted on arXiv 22 Sep 2025.
Builds on
7 of 40 references.
References · 7
- 2020On Layer Normalization in the Transformer Architecturecited by 922
- 2018Attention, Learn to Solve Routing Problems!cited by 656
- 2019Stabilizing Transformers for Reinforcement Learningcited by 270
- 2023PyVRP: a high-performance VRP solver packagecited by 52
- 2017Limits of End-to-End Learningcited by 35
- 2024Scenario Predict-then-Optimize for Data-Driven Online Inventory Routingcited by 7
- 2022Reinforcement Learning Approaches for the Orienteering Problem with Stochastic and Dynamic Release Datescited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2021An Offline Deep Reinforcement Learning for Maintenance Decision-Making
- 2024Optimizing Job Shop Scheduling in the Furniture Industry: A Reinforcement Learning Approach Considering Machine Setup, Batch Variability, and Intralogistics
- 2025Structure-Informed Deep Reinforcement Learning for Inventory Management
- 2021Reinforcement Learning based Condition-oriented Maintenance Scheduling for Flow Line Systems
- 2024Scalable Policies for the Dynamic Traveling Multi-Maintainer Problem with Alerts
- 2026Dynamic resource matching in manufacturing using deep reinforcement learning
- 2026Time-Lag-Aware Deep Reinforcement Learning for Flexible Job-Shop Scheduling in PPVC Module Factories
- 2026A Deep Reinforcement Learning Algorithm for the Vehicle Routing Problem with Stochastic Demands and Outsourcing