# Deep Reinforcement Learning for System-on-Chip: Myths and Realities

arXiv: https://arxiv.org/abs/2207.14595 · HTML: https://mlsanity.com/paper/2207.14595/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 51 references that are in the map.
- First posted on arXiv 29 Jul 2022.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [DAG-GNN: DAG Structure Learning with Graph Neural Networks](https://mlsanity.com/paper/1904.10098/) (2019)
- [StarCraft II: A New Challenge for Reinforcement Learning](https://mlsanity.com/paper/1708.04782/) (2017)
- [Learning Scheduling Algorithms for Data Processing Clusters](https://mlsanity.com/paper/1810.01963/) (2018)
- [Heterogeneity-Aware Cluster Scheduling Policies for Deep Learning Workloads](https://mlsanity.com/paper/2008.09213/) (2020)
- [Themis: Fair and Efficient GPU Cluster Scheduling](https://mlsanity.com/paper/1907.01484/) (2019)
- [Lifelong Learning with a Changing Action Set](https://mlsanity.com/paper/1906.01770/) (2019)
- [DS3: A System-Level Domain-Specific System-on-Chip Simulation Framework](https://mlsanity.com/paper/2003.09016/) (2020)
- [DeepSoCS: A Neural Scheduler for Heterogeneous System-on-Chip (SoC) Resource Scheduling](https://mlsanity.com/paper/2005.07666/) (2020)
- [Neural Heterogeneous Scheduler](https://mlsanity.com/paper/1906.03724/) (2019)
- [A Scalable and Reproducible System-on-Chip Simulation for Reinforcement Learning](https://mlsanity.com/paper/2104.13187/) (2021)
- [SoCRATES: System-on-Chip Resource Adaptive Scheduling using Deep Reinforcement Learning](https://mlsanity.com/paper/2104.14354/) (2021)
- [DAS: Dynamic Adaptive Scheduling for Energy-Efficient Heterogeneous SoCs](https://mlsanity.com/paper/2109.11069/) (2021)
- [Performant, Multi-objective Scheduling of Highly Interleaved Task Graphs on Heterogeneous System on Chip Devices](https://mlsanity.com/paper/2112.08980/) (2021)
- [CEDR -- A Compiler-integrated, Extensible DSSoC Runtime](https://mlsanity.com/paper/2204.08962/) (2022)

## Abstract (arXiv)

Neural schedulers based on deep reinforcement learning (DRL) have shown considerable potential for solving real-world resource allocation problems, as they have demonstrated significant performance gain in the domain of cluster computing. In this paper, we investigate the feasibility of neural schedulers for the domain of System-on-Chip (SoC) resource allocation through extensive experiments and comparison with non-neural, heuristic schedulers. The key finding is three-fold. First, neural schedulers designed for cluster computing domain do not work well for SoC due to i) heterogeneity of SoC computing resources and ii) variable action set caused by randomness in incoming jobs. Second, our novel neural scheduler technique, Eclectic Interaction Matching (EIM), overcomes the above challenges, thus significantly improving the existing neural schedulers. Specifically, we rationalize the underlying reasons behind the performance gain by the EIM-based neural scheduler. Third, we discover that the ratio of the average processing elements (PE) switching delay and the average PE computation time significantly impacts the performance of neural SoC schedulers even with EIM. Consequently, future neural SoC scheduler design must consider this metric as well as its implementation overhead for practical utility.