# A Framework for dynamically meeting performance objectives on a service mesh

arXiv: https://arxiv.org/abs/2306.14178 · HTML: https://mlsanity.com/paper/2306.14178/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 82 references that are in the map.
- First posted on arXiv 25 Jun 2023.

## Builds on (most cited first, 8 of 8)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [A Closer Look at Invalid Action Masking in Policy Gradient Algorithms](https://mlsanity.com/paper/2006.14171/) (2020)
- [FIRM: An Intelligent Fine-Grained Resource Management Framework for SLO-Oriented Microservices](https://mlsanity.com/paper/2008.08509/) (2020)
- [Task Placement and Resource Allocation for Edge Machine Learning: A GNN-based Multi-Agent Reinforcement Learning Paradigm](https://mlsanity.com/paper/2302.00571/) (2023)
- [A Deep Recurrent Q Network towards Self-adapting Distributed Microservices architecture](https://mlsanity.com/paper/1901.04011/) (2019)
- [Heuristic and Reinforcement Learning Algorithms for Dynamic Service Placement on Mobile Edge Cloud](https://mlsanity.com/paper/2111.00240/) (2021)
- [SQLR: Short-Term Memory Q-Learning for Elastic Provisioning](https://mlsanity.com/paper/1909.05772/) (2019)
- [Dynamically meeting performance objectives for multiple services on a service mesh](https://mlsanity.com/paper/2210.04002/) (2022)

## Abstract (arXiv)

We present a framework for achieving end-to-end management objectives for multiple services that concurrently execute on a service mesh. We apply reinforcement learning (RL) techniques to train an agent that periodically performs control actions to reallocate resources. We develop and evaluate the framework using a laboratory testbed where we run information and computing services on a service mesh, supported by the Istio and Kubernetes platforms. We investigate different management objectives that include end-to-end delay bounds on service requests, throughput objectives, cost-related objectives, and service differentiation. We compute the control policies on a simulator rather than on the testbed, which speeds up the training time by orders of magnitude for the scenarios we study. Our proposed framework is novel in that it advocates a top-down approach whereby the management objectives are defined first and then mapped onto the available control actions. It allows us to execute several types of control actions simultaneously. By first learning the system model and the operating region from testbed traces, we can train the agent for different management objectives in parallel.