# Constrained Reinforcement Learning for Adaptive Controller Synchronization in Distributed SDN

arXiv: https://arxiv.org/abs/2403.08775 · HTML: https://mlsanity.com/paper/2403.08775/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 8 of its 25 references that are in the map.
- First posted on arXiv 21 Jan 2024.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 8 of 8)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [DQ Scheduler: Deep Reinforcement Learning Based Controller Synchronization in Distributed SDN](https://mlsanity.com/paper/1812.00852/) (2018)
- [MACS: Deep Reinforcement Learning based SDN Controller Synchronization Policy Design](https://mlsanity.com/paper/1909.09063/) (2019)
- [Learning the Optimal Synchronization Rates in Distributed SDN Control Architectures](https://mlsanity.com/paper/1901.08936/) (2019)
- [Impact of Adaptive Consistency on Distributed SDN Applications: An Empirical Study](https://mlsanity.com/paper/1902.02543/) (2019)
- [Joint SDN Synchronization and Controller Placement in Wireless Networks using Deep Reinforcement Learning](https://mlsanity.com/paper/2311.05582/) (2023)

## Abstract (arXiv)

In software-defined networking (SDN), the implementation of distributed SDN controllers, with each controller responsible for managing a specific sub-network or domain, plays a critical role in achieving a balance between centralized control, scalability, reliability, and network efficiency. These controllers must be synchronized to maintain a logically centralized view of the entire network. While there are various approaches for synchronizing distributed SDN controllers, most tend to prioritize goals such as optimization of communication latency or load balancing, often neglecting to address both the aspects simultaneously. This limitation becomes particularly significant when considering applications like Augmented and Virtual Reality (AR/VR), which demand constrained network latencies and substantial computational resources. Additionally, many existing studies in this field predominantly rely on value-based reinforcement learning (RL) methods, overlooking the potential advantages offered by state-of-the-art policy-based RL algorithms. To bridge this gap, our work focuses on examining deep reinforcement learning (DRL) techniques, encompassing both value-based and policy-based methods, to guarantee an upper latency threshold for AR/VR task offloading within SDN environments, while selecting the most cost-effective servers for AR/VR task offloading. Our evaluation results indicate that while value-based methods excel in optimizing individual network metrics such as latency or load balancing, policy-based approaches exhibit greater robustness in adapting to sudden network changes or reconfiguration.