# Symbolic Distillation for Learned TCP Congestion Control

arXiv: https://arxiv.org/abs/2210.16987 · HTML: https://mlsanity.com/paper/2210.16987/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 56 references that are in the map.
- First posted on arXiv 24 Oct 2022.

## Builds on (most cited first, 19 of 19)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Discovering Symbolic Models from Deep Learning with Inductive Biases](https://mlsanity.com/paper/2006.11287/) (2020)
- [Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients](https://mlsanity.com/paper/1912.04871/) (2019)
- [Distral: Robust Multitask Reinforcement Learning](https://mlsanity.com/paper/1707.04175/) (2017)
- [End-to-end symbolic regression with transformers](https://mlsanity.com/paper/2204.10532/) (2022)
- [Explainable Reinforcement Learning Through a Causal Lens](https://mlsanity.com/paper/1905.10958/) (2019)
- [Cold Brew: Distilling Graph Node Representations with Incomplete or Missing Neighborhoods](https://mlsanity.com/paper/2111.04840/) (2021)
- [Is Label Smoothing Truly Incompatible with Knowledge Distillation: An Empirical Study](https://mlsanity.com/paper/2104.00676/) (2021)
- [Interestingness Elements for Explainable Reinforcement Learning: Understanding Agents' Capabilities and Limitations](https://mlsanity.com/paper/1912.09007/) (2019)
- [Divide-and-Conquer Reinforcement Learning](https://mlsanity.com/paper/1711.09874/) (2017)
- [Generation of Policy-Level Explanations for Reinforcement Learning](https://mlsanity.com/paper/1905.12044/) (2019)
- [Training Your Sparse Neural Network Better with Any Mask](https://mlsanity.com/paper/2206.12755/) (2022)
- [Symbolic Learning to Optimize: Towards Interpretability and Scalability](https://mlsanity.com/paper/2203.06578/) (2022)
- [Towards Symbolic Reinforcement Learning with Common Sense](https://mlsanity.com/paper/1804.08597/) (2018)
- [Symbolic Regression Methods for Reinforcement Learning](https://mlsanity.com/paper/1903.09688/) (2019)
- [Planning from Pixels in Atari with Learned Symbolic Representations](https://mlsanity.com/paper/2012.09126/) (2020)
- [Explaining Deep Reinforcement Learning Agents In The Atari Domain through a Surrogate Model](https://mlsanity.com/paper/2110.03184/) (2021)
- [Self-Supervised Discovering of Interpretable Features for Reinforcement Learning](https://mlsanity.com/paper/2003.07069/) (2020)

## Abstract (arXiv)

Recent advances in TCP congestion control (CC) have achieved tremendous success with deep reinforcement learning (RL) approaches, which use feedforward neural networks (NN) to learn complex environment conditions and make better decisions. However, such "black-box" policies lack interpretability and reliability, and often, they need to operate outside the traditional TCP datapath due to the use of complex NNs. This paper proposes a novel two-stage solution to achieve the best of both worlds: first to train a deep RL agent, then distill its (over-)parameterized NN policy into white-box, light-weight rules in the form of symbolic expressions that are much easier to understand and to implement in constrained environments. At the core of our proposal is a novel symbolic branching algorithm that enables the rule to be aware of the context in terms of various network conditions, eventually converting the NN policy into a symbolic tree. The distilled symbolic rules preserve and often improve performance over state-of-the-art NN policies while being faster and simpler than a standard neural network. We validate the performance of our distilled symbolic rules on both simulation and emulation environments. Our code is available at https://github.com/VITA-Group/SymbolicPCC.