# VeRLPy: Python Library for Verification of Digital Designs with Reinforcement Learning

arXiv: https://arxiv.org/abs/2108.03978 · HTML: https://mlsanity.com/paper/2108.03978/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 22 references that are in the map.
- First posted on arXiv 9 Aug 2021.

## Builds on (most cited first, 13 of 13)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [Hindsight Experience Replay](https://mlsanity.com/paper/1707.01495/) (2017)
- [Deep Reinforcement Learning in Large Discrete Action Spaces](https://mlsanity.com/paper/1512.07679/) (2015)
- [Chip Placement with Deep Reinforcement Learning](https://mlsanity.com/paper/2004.10746/) (2020)
- [Machine Learning for Electronic Design Automation: A Survey](https://mlsanity.com/paper/2102.03357/) (2021)
- [A Hierarchical Framework of Cloud Resource Allocation and Power Management Using Deep Reinforcement Learning](https://mlsanity.com/paper/1703.04221/) (2017)
- [Optimizing Design Verification using Machine Learning: Doing better than Random](https://mlsanity.com/paper/1909.13168/) (2019)
- [Placement Optimization with Deep Reinforcement Learning](https://mlsanity.com/paper/2003.08445/) (2020)
- [Deep Reinforcement Learning-based UAV Navigation and Control: A Soft Actor-Critic with Hindsight Experience Replay Approach](https://mlsanity.com/paper/2106.01016/) (2021)
- [A New Approach for Resource Scheduling with Deep Reinforcement Learning](https://mlsanity.com/paper/1806.08122/) (2018)

## Built on by (most cited first, 3 of 3)

- [Efficient Stimuli Generation using Reinforcement Learning in Design Verification](https://mlsanity.com/paper/2405.19815/) (2024)
- [A Benchmark on Directed Graph Representation Learning in Hardware Designs](https://mlsanity.com/paper/2410.06460/) (2024)
- [Review of Machine Learning for Micro-Electronic Design Verification](https://mlsanity.com/paper/2503.11687/) (2025)

## Abstract (arXiv)

Digital hardware is verified by comparing its behavior against a reference model on a range of randomly generated input signals. The random generation of the inputs hopes to achieve sufficient coverage of the different parts of the design. However, such coverage is often difficult to achieve, amounting to large verification efforts and delays. An alternative is to use Reinforcement Learning (RL) to generate the inputs by learning to prioritize those inputs which can more efficiently explore the design under test. In this work, we present VeRLPy an open-source library to allow RL-driven verification with limited additional engineering overhead. This contributes to two broad movements within the EDA community of (a) moving to open-source toolchains and (b) reducing barriers for development with Python support. We also demonstrate the use of VeRLPy for a few designs and establish its value over randomly generated input signals.