# Regulatory DNA sequence Design with Reinforcement Learning

arXiv: https://arxiv.org/abs/2503.07981 · HTML: https://mlsanity.com/paper/2503.07981/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 64 references that are in the map.
- First posted on arXiv 11 Mar 2025.

## Builds on (most cited first, 19 of 19)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Statistical Rejection Sampling Improves Preference Optimization](https://mlsanity.com/paper/2309.06657/) (2023)
- [HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution](https://mlsanity.com/paper/2306.15794/) (2023)
- [Biological Sequence Design with GFlowNets](https://mlsanity.com/paper/2203.04115/) (2022)
- [Motif-based Graph Self-Supervised Learning for Molecular Property Prediction](https://mlsanity.com/paper/2110.00987/) (2021)
- [Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modeling](https://mlsanity.com/paper/2403.03234/) (2024)
- [Dirichlet Diffusion Score Model for Biological Sequence Generation](https://mlsanity.com/paper/2305.10699/) (2023)
- [AdaLead: A simple and robust adaptive greedy search algorithm for sequence design](https://mlsanity.com/paper/2010.02141/) (2020)
- [De Novo Molecular Generation via Connection-aware Motif Mining](https://mlsanity.com/paper/2302.01129/) (2023)
- [Dynamic Prompt Optimizing for Text-to-Image Generation](https://mlsanity.com/paper/2404.04095/) (2024)
- [Bridging Model-Based Optimization and Generative Modeling via Conservative Fine-Tuning of Diffusion Models](https://mlsanity.com/paper/2405.19673/) (2024)
- [Searching for High-Value Molecules Using Reinforcement Learning and Transformers](https://mlsanity.com/paper/2310.02902/) (2023)
- [VQDNA: Unleashing the Power of Vector Quantization for Multi-Species Genomic Sequence Modeling](https://mlsanity.com/paper/2405.10812/) (2024)
- [Robust Optimization in Protein Fitness Landscapes Using Reinforcement Learning in Latent Space](https://mlsanity.com/paper/2405.18986/) (2024)
- [DiscDiff: Latent Diffusion Model for DNA Sequence Generation](https://mlsanity.com/paper/2402.06079/) (2024)
- [Fast differentiable DNA and protein sequence optimization for molecular design](https://mlsanity.com/paper/2005.11275/) (2020)

## Built on by (most cited first, 6 of 6)

- [D3LM: A Discrete DNA Diffusion Language Model for Bidirectional DNA Understanding and Generation](https://mlsanity.com/paper/2603.01780/) (2026)
- [Ctrl-DNA: Controllable Cell-Type-Specific Regulatory DNA Design via Constrained RL](https://mlsanity.com/paper/2505.20578/) (2025)
- [SPACE: Your Genomic Profile Predictor is a Powerful DNA Foundation Model](https://mlsanity.com/paper/2506.01833/) (2025)
- [Discrete Flow Matching Policy Optimization](https://mlsanity.com/paper/2604.06491/) (2026)
- [Beyond Conditional Computation: Retrieval-Augmented Genomic Foundation Models with Gengram](https://mlsanity.com/paper/2601.22203/) (2026)
- [Extending Sequence Length is Not All You Need: Effective Integration of Multimodal Signals for Gene Expression Prediction](https://mlsanity.com/paper/2602.21550/) (2026)

## Abstract (arXiv)

Cis-regulatory elements (CREs), such as promoters and enhancers, are relatively short DNA sequences that directly regulate gene expression. The fitness of CREs, measured by their ability to modulate gene expression, highly depends on the nucleotide sequences, especially specific motifs known as transcription factor binding sites (TFBSs). Designing high-fitness CREs is crucial for therapeutic and bioengineering applications. Current CRE design methods are limited by two major drawbacks: (1) they typically rely on iterative optimization strategies that modify existing sequences and are prone to local optima, and (2) they lack the guidance of biological prior knowledge in sequence optimization. In this paper, we address these limitations by proposing a generative approach that leverages reinforcement learning (RL) to fine-tune a pre-trained autoregressive (AR) model. Our method incorporates data-driven biological priors by deriving computational inference-based rewards that simulate the addition of activator TFBSs and removal of repressor TFBSs, which are then integrated into the RL process. We evaluate our method on promoter design tasks in two yeast media conditions and enhancer design tasks for three human cell types, demonstrating its ability to generate high-fitness CREs while maintaining sequence diversity. The code is available at https://github.com/yangzhao1230/TACO.