# Model-based Reinforcement Learning for Service Mesh Fault Resiliency in a Web Application-level

arXiv: https://arxiv.org/abs/2110.13621 · HTML: https://mlsanity.com/paper/2110.13621/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 51 references that are in the map.
- First posted on arXiv 21 Oct 2021.

## Builds on (most cited first, 20 of 20)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Model-Based Reinforcement Learning for Atari](https://mlsanity.com/paper/1903.00374/) (2019)
- [Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning](https://mlsanity.com/paper/1708.02596/) (2017)
- [Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents](https://mlsanity.com/paper/1802.08757/) (2018)
- [Learning Attentional Communication for Multi-Agent Cooperation](https://mlsanity.com/paper/1805.07733/) (2018)
- [Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Control](https://mlsanity.com/paper/1903.04527/) (2019)
- [Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guarantees](https://mlsanity.com/paper/1807.03858/) (2018)
- [Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1702.08887/) (2017)
- [Learning to Schedule Communication in Multi-agent Reinforcement Learning](https://mlsanity.com/paper/1902.01554/) (2019)
- [On the role of planning in model-based deep reinforcement learning](https://mlsanity.com/paper/2011.04021/) (2020)
- [Simplified Action Decoder for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1912.02288/) (2019)
- [Biases for Emergent Communication in Multi-agent Reinforcement Learning](https://mlsanity.com/paper/1912.05676/) (2019)
- [No Press Diplomacy: Modeling Multi-Agent Gameplay](https://mlsanity.com/paper/1909.02128/) (2019)
- [Lenient Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/1707.04402/) (2017)
- [Decentralized Multi-Agent Reinforcement Learning with Networked Agents: Recent Advances](https://mlsanity.com/paper/1912.03821/) (2019)
- [ACCNet: Actor-Coordinator-Critic Net for "Learning-to-Communicate" with Deep Multi-agent Reinforcement Learning](https://mlsanity.com/paper/1706.03235/) (2017)
- [Federated Control with Hierarchical Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/1712.08266/) (2017)
- [Collaboration of AI Agents via Cooperative Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/1907.00327/) (2019)
- [Self-Adaptive Microservice-based Systems -- Landscape and Research Opportunities](https://mlsanity.com/paper/2103.08688/) (2021)

## Built on by (most cited first, 5 of 5)

- [Evolution and Efficiency in Neural Architecture Search: Bridging the Gap Between Expert Design and Automated Optimization](https://mlsanity.com/paper/2403.17012/) (2024)
- [FedEmb: A Vertical and Hybrid Federated Learning Algorithm using Network And Feature Embedding Aggregation](https://mlsanity.com/paper/2312.00102/) (2023)
- [Joint Detection Algorithm for Multiple Cognitive Users in Spectrum Sensing](https://mlsanity.com/paper/2311.18599/) (2023)
- [Sample-based Dynamic Hierarchical Transformer with Layer and Head Flexibility via Contextual Bandit](https://mlsanity.com/paper/2312.03038/) (2023)
- [Optimizing the Passenger Flow for Airport Security Check](https://mlsanity.com/paper/2312.05259/) (2023)

## Abstract (arXiv)

Microservice-based architectures enable different aspects of web applications to be created and updated independently, even after deployment. Associated technologies such as service mesh provide application-level fault resilience through attribute configurations that govern the behavior of request-response service -- and the interactions among them -- in the presence of failures. While this provides tremendous flexibility, the configured values of these attributes -- and the relationships among them -- can significantly affect the performance and fault resilience of the overall application. Furthermore, it is impossible to determine the best and worst combinations of attribute values with respect to fault resiliency via testing, due to the complexities of the underlying distributed system and the many possible attribute value combinations. In this paper, we present a model-based reinforcement learning workflow towards service mesh fault resiliency. Our approach enables the prediction of the most significant fault resilience behaviors at a web application-level, scratching from single service to aggregated multi-service management with efficient agent collaborations.