# BEACON: Benchmark for Comprehensive RNA Tasks and Language Models

arXiv: https://arxiv.org/abs/2406.10391 · HTML: https://mlsanity.com/paper/2406.10391/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 123 references that are in the map.
- First posted on arXiv 14 Jun 2024.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation](https://mlsanity.com/paper/2108.12409/) (2021)
- [Learning from Protein Structure with Geometric Vector Perceptrons](https://mlsanity.com/paper/2009.01411/) (2020)
- [Evaluating Protein Transfer Learning with TAPE](https://mlsanity.com/paper/1906.08230/) (2019)
- [DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genome](https://mlsanity.com/paper/2306.15006/) (2023)
- [PiFold: Toward effective and efficient protein inverse folding](https://mlsanity.com/paper/2209.12643/) (2022)
- [PEER: A Comprehensive and Multi-Task Benchmark for Protein Sequence Understanding](https://mlsanity.com/paper/2206.02096/) (2022)
- [Learning to Design RNA](https://mlsanity.com/paper/1812.11951/) (2018)
- [BEND: Benchmarking DNA Language Models on biologically meaningful tasks](https://mlsanity.com/paper/2311.12570/) (2023)
- [Is Transfer Learning Necessary for Protein Landscape Prediction?](https://mlsanity.com/paper/2011.03443/) (2020)
- [RNA Secondary Structure Prediction By Learning Unrolled Algorithms](https://mlsanity.com/paper/2002.05810/) (2020)
- [gRNAde: Geometric Deep Learning for 3D RNA inverse design](https://mlsanity.com/paper/2305.14749/) (2023)
- [A 5' UTR Language Model for Decoding Untranslated Regions of mRNA and Function Predictions](https://mlsanity.com/paper/2310.03281/) (2023)
- [LinearFold: linear-time approximate RNA folding by 5'-to-3' dynamic programming and beam search](https://mlsanity.com/paper/2001.04020/) (2019)
- [ncRNA Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1905.06515/) (2019)
- [Deep learning models for predicting RNA degradation via dual crowdsourcing](https://mlsanity.com/paper/2110.07531/) (2021)
- [Linguistically inspired roadmap for building biologically reliable protein language models](https://mlsanity.com/paper/2207.00982/) (2022)

## Built on by (most cited first, 7 of 7)

- [LLM-based HSE Compliance Assessment: Benchmark, Performance, and Advancements](https://mlsanity.com/paper/2505.22959/) (2025)
- [Benchmarking Large Language Models on Multiple Tasks in Bioinformatics NLP with Prompting](https://mlsanity.com/paper/2503.04013/) (2025)
- [COMET: Benchmark for Comprehensive Biological Multi-omics Evaluation Tasks and Language Models](https://mlsanity.com/paper/2412.10347/) (2024)
- [Character-level Tokenizations as Powerful Inductive Biases for RNA Foundational Models](https://mlsanity.com/paper/2411.11808/) (2024)
- [A Comprehensive Benchmark for RNA 3D Structure-Function Modeling](https://mlsanity.com/paper/2503.21681/) (2025)
- [OmniGenBench: Automating Large-scale in-silico Benchmarking for Genomic Foundation Models](https://mlsanity.com/paper/2410.01784/) (2024)
- [MultiMolecule: a modular ecosystem for biomolecular sequence-model workflows](https://mlsanity.com/paper/2606.16540/) (2026)

## Abstract (arXiv)

RNA plays a pivotal role in translating genetic instructions into functional outcomes, underscoring its importance in biological processes and disease mechanisms. Despite the emergence of numerous deep learning approaches for RNA, particularly universal RNA language models, there remains a significant lack of standardized benchmarks to assess the effectiveness of these methods. In this study, we introduce the first comprehensive RNA benchmark BEACON (BEnchmArk for COmprehensive RNA Task and Language Models). First, BEACON comprises 13 distinct tasks derived from extensive previous work covering structural analysis, functional studies, and engineering applications, enabling a comprehensive assessment of the performance of methods on various RNA understanding tasks. Second, we examine a range of models, including traditional approaches like CNNs, as well as advanced RNA foundation models based on language models, offering valuable insights into the task-specific performances of these models. Third, we investigate the vital RNA language model components from the tokenizer and positional encoding aspects. Notably, our findings emphasize the superiority of single nucleotide tokenization and the effectiveness of Attention with Linear Biases (ALiBi) over traditional positional encoding methods. Based on these insights, a simple yet strong baseline called BEACON-B is proposed, which can achieve outstanding performance with limited data and computational resources. The datasets and source code of our benchmark are available at https://github.com/terry-r123/RNABenchmark.