# TrojanForge: Generating Adversarial Hardware Trojan Examples Using Reinforcement Learning

arXiv: https://arxiv.org/abs/2405.15184 · HTML: https://mlsanity.com/paper/2405.15184/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods.
- Builds on 9 of its 26 references that are in the map.
- First posted on arXiv 24 May 2024.

## Placed in

- Deep reinforcement learning methods / [Adversarial robustness and RL in finance](https://mlsanity.com/deeprl/adversarial-robustness-and-rl-in-finance/)

## Builds on (most cited first, 9 of 9)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [ATTRITION: Attacking Static Hardware Trojan Detection Techniques Using Reinforcement Learning](https://mlsanity.com/paper/2208.12897/) (2022)
- [Hardware Trojan Insertion Using Reinforcement Learning](https://mlsanity.com/paper/2204.04350/) (2022)
- [AttackGNN: Red-Teaming GNNs in Hardware Security Using Reinforcement Learning](https://mlsanity.com/paper/2402.13946/) (2024)
- [DETERRENT: Detecting Trojans using Reinforcement Learning](https://mlsanity.com/paper/2208.12878/) (2022)
- [Trojan Playground: A Reinforcement Learning Framework for Hardware Trojan Insertion and Detection](https://mlsanity.com/paper/2305.09592/) (2023)
- [Multi-criteria Hardware Trojan Detection: A Reinforcement Learning Approach](https://mlsanity.com/paper/2304.13232/) (2023)
- [R-HTDetector: Robust Hardware-Trojan Detection Based on Adversarial Training](https://mlsanity.com/paper/2205.13702/) (2022)
- [The Seeker's Dilemma: Realistic Formulation and Benchmarking for Hardware Trojan Detection](https://mlsanity.com/paper/2402.17918/) (2024)

## Abstract (arXiv)

The Hardware Trojan (HT) problem can be thought of as a continuous game between attackers and defenders, each striving to outsmart the other by leveraging any available means for an advantage. Machine Learning (ML) has recently played a key role in advancing HT research. Various novel techniques, such as Reinforcement Learning (RL) and Graph Neural Networks (GNNs), have shown HT insertion and detection capabilities. HT insertion with ML techniques, specifically, has seen a spike in research activity due to the shortcomings of conventional HT benchmarks and the inherent human design bias that occurs when we create them. This work continues this innovation by presenting a tool called TrojanForge, capable of generating HT adversarial examples that defeat HT detectors; demonstrating the capabilities of GAN-like adversarial tools for automatic HT insertion. We introduce an RL environment where the RL insertion agent interacts with HT detectors in an insertion-detection loop where the agent collects rewards based on its success in bypassing HT detectors. Our results show that this process helps inserted HTs evade various HT detectors, achieving high attack success percentages. This tool provides insight into why HT insertion fails in some instances and how we can leverage this knowledge in defense.