# Age of Information Aware VNF Scheduling in Industrial IoT Using Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2105.04207 · HTML: https://mlsanity.com/paper/2105.04207/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 44 references that are in the map.
- First posted on arXiv 10 May 2021.

## Builds on (most cited first, 14 of 14)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [An Introduction to Deep Reinforcement Learning](https://mlsanity.com/paper/1811.12560/) (2018)
- [Deep Reinforcement Learning for Resource Management in Network Slicing](https://mlsanity.com/paper/1805.06591/) (2018)
- [Deep Learning based Wireless Resource Allocation with Application to Vehicular Networks](https://mlsanity.com/paper/1907.03289/) (2019)
- [Cooperative Internet of UAVs: Distributed Trajectory Design by Multi-agent Deep Reinforcement Learning](https://mlsanity.com/paper/2007.14297/) (2020)
- [Joint Status Sampling and Updating for Minimizing Age of Information in the Internet of Things](https://mlsanity.com/paper/1807.04356/) (2018)
- [UAV-to-Device Underlay Communications: Age of Information Minimization by Multi-agent Deep Reinforcement Learning](https://mlsanity.com/paper/2003.05830/) (2020)
- [Optimizing Information Freshness in Computing enabled IoT Networks](https://mlsanity.com/paper/1910.05578/) (2019)
- [CPVNF:Cost-efficient Proactive VNF Placement and Chaining for Value-Added Services in Content Delivery Networks](https://mlsanity.com/paper/1803.06025/) (2018)
- [Minimum Age of Information in the Internet of Things with Non-uniform Status Packet Sizes](https://mlsanity.com/paper/1901.07069/) (2019)
- [Minimizing Age of Information for Real-Time Monitoring in Resource-Constrained Industrial IoT Networks](https://mlsanity.com/paper/1912.07186/) (2019)
- [Extendable NFV-Integrated Control Method Using Reinforcement Learning](https://mlsanity.com/paper/1912.09022/) (2019)
- [Age-Optimal Power Allocation in Industrial IoT: A Risk-Sensitive Federated Learning Approach](https://mlsanity.com/paper/2012.06860/) (2020)

## Built on by (most cited first, 8 of 8)

- [Latency Optimization for Blockchain-Empowered Federated Learning in Multi-Server Edge Computing](https://mlsanity.com/paper/2203.09670/) (2022)
- [Multi-Agent Deep Reinforcement Learning for Cost- and Delay-Sensitive Virtual Network Function Placement and Routing](https://mlsanity.com/paper/2206.12146/) (2022)
- [Deep Reinforcement Learning for Task Offloading in UAV-Aided Smart Farm Networks](https://mlsanity.com/paper/2209.07367/) (2022)
- [IoT-Aerial Base Station Task Offloading with Risk-Sensitive Reinforcement Learning for Smart Agriculture](https://mlsanity.com/paper/2209.07382/) (2022)
- [On Jointly Optimizing Partial Offloading and SFC Mapping: A Cooperative Dual-agent Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2205.09925/) (2022)
- [Semantic-aware Transmission Scheduling: a Monotonicity-driven Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2305.13706/) (2023)
- [How to Minimize the Weighted Sum AoI in Multi-Source Status Update Systems: OMA or NOMA?](https://mlsanity.com/paper/2205.03143/) (2022)
- [Asynchronous Fractional Multi-Agent Deep Reinforcement Learning for Age-Minimal Mobile Edge Computing](https://mlsanity.com/paper/2409.16832/) (2024)

## Abstract (arXiv)

In delay-sensitive industrial internet of things (IIoT) applications, the age of information (AoI) is employed to characterize the freshness of information. Meanwhile, the emerging network function virtualization provides flexibility and agility for service providers to deliver a given network service using a sequence of virtual network functions (VNFs). However, suitable VNF placement and scheduling in these schemes is NP-hard and finding a globally optimal solution by traditional approaches is complex. Recently, deep reinforcement learning (DRL) has appeared as a viable way to solve such problems. In this paper, we first utilize single agent low-complex compound action actor-critic RL to cover both discrete and continuous actions and jointly minimize VNF cost and AoI in terms of network resources under end-to end Quality of Service constraints. To surmount the single-agent capacity limitation for learning, we then extend our solution to a multi-agent DRL scheme in which agents collaborate with each other. Simulation results demonstrate that single-agent schemes significantly outperform the greedy algorithm in terms of average network cost and AoI. Moreover, multi-agent solution decreases the average cost by dividing the tasks between the agents. However, it needs more iterations to be learned due to the requirement on the agents collaboration.