# Designing RNAs with Language Models

arXiv: https://arxiv.org/abs/2602.12470 · HTML: https://mlsanity.com/paper/2602.12470/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 58 references that are in the map.
- First posted on arXiv 12 Feb 2026.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Improving Neural Machine Translation Models with Monolingual Data](https://mlsanity.com/paper/1511.06709/) (2015)
- [TTRL: Test-Time Reinforcement Learning](https://mlsanity.com/paper/2504.16084/) (2025)
- [XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Models](https://mlsanity.com/paper/2411.15100/) (2024)
- [RiNALMo: General-Purpose RNA Language Models Can Generalize Well on Structure Prediction Tasks](https://mlsanity.com/paper/2403.00043/) (2024)
- [Learning to Design RNA](https://mlsanity.com/paper/1812.11951/) (2018)
- [Designing RNA Secondary Structures is Hard](https://mlsanity.com/paper/1710.11513/) (2017)
- [When to Finish? Optimal Beam Search for Neural Text Generation (modulo beam size)](https://mlsanity.com/paper/1809.00069/) (2018)
- [RNA-GPT: Multimodal Generative System for RNA Sequence Understanding](https://mlsanity.com/paper/2411.08900/) (2024)
- [Comprehensive benchmarking of large language models for RNA secondary structure prediction](https://mlsanity.com/paper/2410.16212/) (2024)
- [Efficient Encoder-Decoder Transformer Decoding for Decomposable Tasks](https://mlsanity.com/paper/2403.13112/) (2024)

## Built on by (most cited first, 1 of 1)

- [RNA Design via Conditioned Flow Matching and Finite-Policy Reinforcement Learning](https://mlsanity.com/paper/2609.36885/) (2026)

## Abstract (arXiv)

RNA design, the task of finding a sequence that folds into a target secondary structure, has broad biological and biomedical impact but remains computationally challenging due to the exponentially large sequence space and exponentially many competing folds. Traditional approaches treat it as an optimization problem, relying on per-instance heuristics or constraint-based search. We instead reframe RNA design as conditional sequence generation and introduce a reusable neural approximator, instantiated as an autoregressive language model (LM), that maps target structures directly to sequences. We first train our model in a supervised setting on random-induced structure-sequence pairs, and then use reinforcement learning (RL) to optimize end-to-end metrics. We also propose methods to select a small subset for RL that greatly improves RL efficiency and quality. Across four datasets, our approach outperforms state-of-the-art systems on key metrics such as Boltzmann probability while being 1.7x faster, establishing conditional LM generation as a scalable, task-agnostic alternative to per-instance optimization for RNA design. Our code and data are available at https://github.com/KuNyaa/RNA-Design-LM.