# Enhanced Pub/Sub Communications for Massive IoT Traffic with SARSA Reinforcement Learning

arXiv: https://arxiv.org/abs/2101.00687 · HTML: https://mlsanity.com/paper/2101.00687/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 44 references that are in the map.
- First posted on arXiv 3 Jan 2021.

## Builds on (most cited first, 13 of 13)

- [Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents](https://mlsanity.com/paper/1709.06009/) (2017)
- [Better Exploration with Optimistic Actor-Critic](https://mlsanity.com/paper/1910.12807/) (2019)
- [Finite-sample Analysis of Greedy-GQ with Linear Function Approximation under Markovian Noise](https://mlsanity.com/paper/2005.10175/) (2020)
- [OpenEI: An Open Framework for Edge Intelligence](https://mlsanity.com/paper/1906.01864/) (2019)
- [Enabling Cognitive Smart Cities Using Big Data and Machine Learning: Approaches and Challenges](https://mlsanity.com/paper/1810.04107/) (2018)
- [A Short Survey On Memory Based Reinforcement Learning](https://mlsanity.com/paper/1904.06736/) (2019)
- [Deep Reinforcement Learning for Network Slicing with Heterogeneous Resource Requirements and Time Varying Traffic Dynamics](https://mlsanity.com/paper/1908.03242/) (2019)
- [Artificial Intelligence Enabled Software Defined Networking: A Comprehensive Overview](https://mlsanity.com/paper/1803.06818/) (2018)
- [A Pub/Sub SDN-Integrated Framework for IoT Traffic Orchestration](https://mlsanity.com/paper/1907.12505/) (2019)
- [A Comparison of learning algorithms on the Arcade Learning Environment](https://mlsanity.com/paper/1410.8620/) (2014)
- [A Publish/Subscribe QoS-aware Framework for Massive IoT Traffic Orchestration](https://mlsanity.com/paper/1806.03157/) (2018)
- [Towards Smart City Innovation Under the Perspective of Software-Defined Networking, Artificial Intelligence and Big Data](https://mlsanity.com/paper/1810.11665/) (2018)
- [A Reinforcement Learning Approach for the Multichannel Rendezvous Problem](https://mlsanity.com/paper/1907.01919/) (2019)

## Abstract (arXiv)

Sensors are being extensively deployed and are expected to expand at significant rates in the coming years. They typically generate a large volume of data on the internet of things (IoT) application areas like smart cities, intelligent traffic systems, smart grid, and e-health. Cloud, edge and fog computing are potential and competitive strategies for collecting, processing, and distributing IoT data. However, cloud, edge, and fog-based solutions need to tackle the distribution of a high volume of IoT data efficiently through constrained and limited resource network infrastructures. This paper addresses the issue of conveying a massive volume of IoT data through a network with limited communications resources (bandwidth) using a cognitive communications resource allocation based on Reinforcement Learning (RL) with SARSA algorithm. The proposed network infrastructure (PSIoTRL) uses a Publish/ Subscribe architecture to access massive and highly distributed IoT data. It is demonstrated that the PSIoTRL bandwidth allocation for buffer flushing based on SARSA enhances the IoT aggregator buffer occupation and network link utilization. The PSIoTRL dynamically adapts the IoT aggregator traffic flushing according to the Pub/Sub topic's priority and network constraint requirements.