# Experience-driven Networking: A Deep Reinforcement Learning based Approach

arXiv: https://arxiv.org/abs/1801.05757 · HTML: https://mlsanity.com/paper/1801.05757/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jan 2018.

## Built on by (most cited first, 30 of 41)

- [Federated Reinforcement Learning: Techniques, Applications, and Open Challenges](https://mlsanity.com/paper/2108.11887/) (2021)
- [Sample-Efficient Reinforcement Learning with loglog(T) Switching Cost](https://mlsanity.com/paper/2202.06385/) (2022)
- [Graph Neural Networks for Communication Networks: Context, Use Cases and Opportunities](https://mlsanity.com/paper/2112.14792/) (2021)
- [Escaping from Zero Gradient: Revisiting Action-Constrained Reinforcement Learning via Frank-Wolfe Policy Optimization](https://mlsanity.com/paper/2102.11055/) (2021)
- [RouteNet-Fermi: Network Modeling with Graph Neural Networks](https://mlsanity.com/paper/2212.12070/) (2022)
- [Graph Reinforcement Learning for Combinatorial Optimization: A Survey and Unifying Perspective](https://mlsanity.com/paper/2404.06492/) (2024)
- [RouteNet-Erlang: A Graph Neural Network for Network Performance Evaluation](https://mlsanity.com/paper/2202.13956/) (2022)
- [Teal: Learning-Accelerated Optimization of WAN Traffic Engineering](https://mlsanity.com/paper/2210.13763/) (2022)
- [Policy Finetuning in Reinforcement Learning via Design of Experiments using Offline Data](https://mlsanity.com/paper/2307.04354/) (2023)
- [Sample Complexity and Overparameterization Bounds for Temporal Difference Learning with Neural Network Approximation](https://mlsanity.com/paper/2103.01391/) (2021)
- [MAGNNETO: A Graph Neural Network-based Multi-Agent system for Traffic Engineering](https://mlsanity.com/paper/2303.18157/) (2023)
- [Is Machine Learning Ready for Traffic Engineering Optimization?](https://mlsanity.com/paper/2109.01445/) (2021)
- [FENXI: Deep-learning Traffic Analytics at the Edge](https://mlsanity.com/paper/2105.11738/) (2021)
- [The Graph Neural Networking Challenge: A Worldwide Competition for Education in AI/ML for Networks](https://mlsanity.com/paper/2107.12433/) (2021)
- [ENERO: Efficient Real-Time WAN Routing Optimization with Deep Reinforcement Learning](https://mlsanity.com/paper/2109.10883/) (2021)
- [Near-Optimal Adversarial Reinforcement Learning with Switching Costs](https://mlsanity.com/paper/2302.04374/) (2023)
- [A Reinforcement Learning Approach for Scheduling in mmWave Networks](https://mlsanity.com/paper/2108.00548/) (2021)
- [Learning an Adaptive Forwarding Strategy for Mobile Wireless Networks: Resource Usage vs. Latency](https://mlsanity.com/paper/2207.11386/) (2022)
- [Graph Neural Modeling of Network Flows](https://mlsanity.com/paper/2209.05208/) (2022)
- [Towards Safe Load Balancing based on Control Barrier Functions and Deep Reinforcement Learning](https://mlsanity.com/paper/2401.05525/) (2024)
- [RouteNet-Gauss: Hardware-Enhanced Network Modeling with Machine Learning](https://mlsanity.com/paper/2501.08848/) (2025)
- [Action-Constrained Imitation Learning](https://mlsanity.com/paper/2508.14379/) (2025)
- [Time-based Dynamic Controllability of Disjunctive Temporal Networks with Uncertainty: A Tree Search Approach with Graph Neural Network Guidance](https://mlsanity.com/paper/2108.01068/) (2021)
- [MAMRL: Exploiting Multi-agent Meta Reinforcement Learning in WAN Traffic Engineering](https://mlsanity.com/paper/2111.15087/) (2021)
- [Solving Disjunctive Temporal Networks with Uncertainty under Restricted Time-Based Controllability using Tree Search and Graph Neural Networks](https://mlsanity.com/paper/2203.15030/) (2022)
- [Using Deep Reinforcement Learning for mmWave Real-Time Scheduling](https://mlsanity.com/paper/2210.01423/) (2022)
- [FERN: Leveraging Graph Attention Networks for Failure Evaluation and Robust Network Design](https://mlsanity.com/paper/2305.19153/) (2023)
- [Learning State-Augmented Policies for Information Routing in Communication Networks](https://mlsanity.com/paper/2310.00248/) (2023)
- [Safe Load Balancing in Software-Defined-Networking](https://mlsanity.com/paper/2410.16846/) (2024)
- [FAuNO: Semi-Asynchronous Federated Reinforcement Learning Framework for Task Offloading in Edge Systems](https://mlsanity.com/paper/2506.02668/) (2025)

## Abstract (arXiv)

Modern communication networks have become very complicated and highly dynamic, which makes them hard to model, predict and control. In this paper, we develop a novel experience-driven approach that can learn to well control a communication network from its own experience rather than an accurate mathematical model, just as a human learns a new skill (such as driving, swimming, etc). Specifically, we, for the first time, propose to leverage emerging Deep Reinforcement Learning (DRL) for enabling model-free control in communication networks; and present a novel and highly effective DRL-based control framework, DRL-TE, for a fundamental networking problem: Traffic Engineering (TE). The proposed framework maximizes a widely-used utility function by jointly learning network environment and its dynamics, and making decisions under the guidance of powerful Deep Neural Networks (DNNs). We propose two new techniques, TE-aware exploration and actor-critic-based prioritized experience replay, to optimize the general DRL framework particularly for TE. To validate and evaluate the proposed framework, we implemented it in ns-3, and tested it comprehensively with both representative and randomly generated network topologies. Extensive packet-level simulation results show that 1) compared to several widely-used baseline methods, DRL-TE significantly reduces end-to-end delay and consistently improves the network utility, while offering better or comparable throughput; 2) DRL-TE is robust to network changes; and 3) DRL-TE consistently outperforms a state-ofthe-art DRL method (for continuous control), Deep Deterministic Policy Gradient (DDPG), which, however, does not offer satisfying performance.