# Improving RNA Secondary Structure Design using Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2111.04504 · HTML: https://mlsanity.com/paper/2111.04504/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 27 references that are in the map.
- First posted on arXiv 5 Nov 2021.

## Builds on (most cited first, 6 of 6)

- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [MolGAN: An implicit generative model for small molecular graphs](https://mlsanity.com/paper/1805.11973/) (2018)
- [Optimization of Molecules via Deep Reinforcement Learning](https://mlsanity.com/paper/1810.08678/) (2018)
- [Application of generative autoencoder in de novo molecular design](https://mlsanity.com/paper/1711.07839/) (2017)
- [Generating and designing DNA with deep generative models](https://mlsanity.com/paper/1712.06148/) (2017)
- [Bayesian Optimization for Synthetic Gene Design](https://mlsanity.com/paper/1505.01627/) (2015)

## Built on by (most cited first, 2 of 2)

- [Structure in Deep Reinforcement Learning: A Survey and Open Problems](https://mlsanity.com/paper/2306.16021/) (2023)
- [Revolutionizing Genomics with Reinforcement Learning Techniques](https://mlsanity.com/paper/2302.13268/) (2023)

## Abstract (arXiv)

Rising costs in recent years of developing new drugs and treatments have led to extensive research in optimization techniques in biomolecular design. Currently, the most widely used approach in biomolecular design is directed evolution, which is a greedy hill-climbing algorithm that simulates biological evolution. In this paper, we propose a new benchmark of applying reinforcement learning to RNA sequence design, in which the objective function is defined to be the free energy in the sequence's secondary structure. In addition to experimenting with the vanilla implementations of each reinforcement learning algorithm from standard libraries, we analyze variants of each algorithm in which we modify the algorithm's reward function and tune the model's hyperparameters. We show results of the ablation analysis that we do for these algorithms, as well as graphs indicating the algorithm's performance across batches and its ability to search the possible space of RNA sequences. We find that our DQN algorithm performs by far the best in this setting, contrasting with, in which PPO performs the best among all tested algorithms. Our results should be of interest to those in the biomolecular design community and should serve as a baseline for future experiments involving machine learning in molecule design.