# A Scalable and Reproducible System-on-Chip Simulation for Reinforcement Learning

arXiv: https://arxiv.org/abs/2104.13187 · HTML: https://mlsanity.com/paper/2104.13187/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 35 references that are in the map.
- First posted on arXiv 27 Apr 2021.

## Builds on (most cited first, 12 of 12)

- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [Learning Dexterous In-Hand Manipulation](https://mlsanity.com/paper/1808.00177/) (2018)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [Ray: A Distributed Framework for Emerging AI Applications](https://mlsanity.com/paper/1712.05889/) (2017)
- [AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles](https://mlsanity.com/paper/1705.05065/) (2017)
- [Learning Scheduling Algorithms for Data Processing Clusters](https://mlsanity.com/paper/1810.01963/) (2018)
- [Extending the OpenAI Gym for robotics: a toolkit for reinforcement learning using ROS and Gazebo](https://mlsanity.com/paper/1608.05742/) (2016)
- [DS3: A System-Level Domain-Specific System-on-Chip Simulation Framework](https://mlsanity.com/paper/2003.09016/) (2020)
- [DeepSoCS: A Neural Scheduler for Heterogeneous System-on-Chip (SoC) Resource Scheduling](https://mlsanity.com/paper/2005.07666/) (2020)
- [Runtime Task Scheduling using Imitation Learning for Heterogeneous Many-Core Systems](https://mlsanity.com/paper/2007.09361/) (2020)
- [STOMP: A Tool for Evaluation of Scheduling Policies in Heterogeneous Multi-Processors](https://mlsanity.com/paper/2007.14371/) (2020)

## Built on by (most cited first, 1 of 1)

- [Deep Reinforcement Learning for System-on-Chip: Myths and Realities](https://mlsanity.com/paper/2207.14595/) (2022)

## Abstract (arXiv)

Deep Reinforcement Learning (DRL) underlies in a simulated environment and optimizes objective goals. By extending the conventional interaction scheme, this paper proffers gym-ds3, a scalable and reproducible open environment tailored for a high-fidelity Domain-Specific System-on-Chip (DSSoC) application. The simulation corroborates to schedule hierarchical jobs onto heterogeneous System-on-Chip (SoC) processors and bridges the system to reinforcement learning research. We systematically analyze the representative SoC simulator and discuss the primary challenging aspects that the system (1) continuously generates indefinite jobs at a rapid injection rate, (2) optimizes complex objectives, and (3) operates in steady-state scheduling. We provide exemplary snippets and experimentally demonstrate the run-time performances on different schedulers that successfully mimic results achieved from the standard DS3 framework and real-world embedded systems.