# BEND: Benchmarking DNA Language Models on biologically meaningful tasks

arXiv: https://arxiv.org/abs/2311.12570 · HTML: https://mlsanity.com/paper/2311.12570/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 55 references that are in the map.
- First posted on arXiv 21 Nov 2023.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation](https://mlsanity.com/paper/2108.12409/) (2021)
- [Hyena Hierarchy: Towards Larger Convolutional Language Models](https://mlsanity.com/paper/2302.10866/) (2023)
- [Evaluating Protein Transfer Learning with TAPE](https://mlsanity.com/paper/1906.08230/) (2019)
- [Regularizing and Optimizing LSTM Language Models](https://mlsanity.com/paper/1708.02182/) (2017)
- [HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution](https://mlsanity.com/paper/2306.15794/) (2023)
- [DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genome](https://mlsanity.com/paper/2306.15006/) (2023)
- [BERTology Meets Biology: Interpreting Attention in Protein Language Models](https://mlsanity.com/paper/2006.15222/) (2020)
- [PEER: A Comprehensive and Multi-Task Benchmark for Protein Sequence Understanding](https://mlsanity.com/paper/2206.02096/) (2022)
- [Data navigation on the ENCODE portal](https://mlsanity.com/paper/2305.00006/) (2023)

## Built on by (most cited first, 29 of 29)

- [BioReason: Incentivizing Multimodal Biological Reasoning within a DNA-LLM Model](https://mlsanity.com/paper/2505.23579/) (2025)
- [Genomic Language Models: Opportunities and Challenges](https://mlsanity.com/paper/2407.11435/) (2024)
- [VQDNA: Unleashing the Power of Vector Quantization for Multi-Species Genomic Sequence Modeling](https://mlsanity.com/paper/2405.10812/) (2024)
- [GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models](https://mlsanity.com/paper/2406.01627/) (2024)
- [JEPA-DNA: Grounding Genomic Foundation Models through Joint-Embedding Predictive Architectures](https://mlsanity.com/paper/2602.17162/) (2026)
- [BEACON: Benchmark for Comprehensive RNA Tasks and Language Models](https://mlsanity.com/paper/2406.10391/) (2024)
- [HybriDNA: A Hybrid Transformer-Mamba2 Long-Range DNA Language Model](https://mlsanity.com/paper/2502.10807/) (2025)
- [DART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA](https://mlsanity.com/paper/2412.05430/) (2024)
- [SPACE: Your Genomic Profile Predictor is a Powerful DNA Foundation Model](https://mlsanity.com/paper/2506.01833/) (2025)
- [BarcodeMamba: State Space Models for Biodiversity Analysis](https://mlsanity.com/paper/2412.11084/) (2024)
- [Revisiting Convolution Architecture in the Realm of DNA Foundation Models](https://mlsanity.com/paper/2502.18538/) (2025)
- [GV-Rep: A Large-Scale Dataset for Genetic Variant Representation Learning](https://mlsanity.com/paper/2407.16940/) (2024)
- [A Phylogenetic Approach to Genomic Language Modeling](https://mlsanity.com/paper/2503.03773/) (2025)
- [Same model, better performance: the impact of shuffling on DNA Language Models benchmarking](https://mlsanity.com/paper/2510.12617/) (2025)
- [How Private Are DNA Embeddings? Inverting Foundation Model Representations of Genomic Sequences](https://mlsanity.com/paper/2603.06950/) (2026)
- [GenomeQA: Benchmarking General Large Language Models for Genome Sequence Understanding](https://mlsanity.com/paper/2604.05774/) (2026)
- [PhageBench: Can LLMs Understand Raw Bacteriophage Genomes?](https://mlsanity.com/paper/2604.05775/) (2026)
- [Evaluating Post-hoc Explanations of the Transformer-based Genome Language Model DNABERT-2](https://mlsanity.com/paper/2604.21690/) (2026)
- [M5: A Whole Genome Bacterial Encoder at Single Nucleotide Resolution](https://mlsanity.com/paper/2407.03392/) (2024)
- [Recent advances in deep learning and language models for studying the microbiome](https://mlsanity.com/paper/2409.10579/) (2024)
- [DNACHUNKER: Learnable Tokenization for DNA Language Models](https://mlsanity.com/paper/2601.03019/) (2026)
- [Rethinking Genomic Modeling Through Optical Character Recognition](https://mlsanity.com/paper/2602.02014/) (2026)
- [In Search of Lost DNA Sequence Pretraining](https://mlsanity.com/paper/2604.16570/) (2026)
- [Wisteria: A Unified Multi-Scale Feature Learning Framework for DNA Language Model](https://mlsanity.com/paper/2605.05913/) (2026)
- [Bridging Sequence and Graph Structure for Epigenetic Age Prediction](https://mlsanity.com/paper/2605.10541/) (2026)
- [ViroBench: Benchmarking Nucleotide Foundation Models on Viral Genomics Tasks](https://mlsanity.com/paper/2605.25388/) (2026)
- [GENEB: Why Genomic Models Are Hard to Compare](https://mlsanity.com/paper/2606.04525/) (2026)
- [TheBioCollection: Unified Pre-Training Scale LLM Corpus for Biology](https://mlsanity.com/paper/2607.08803/) (2026)
- [What DNA Foundation Models Learn Beyond Sequence Composition](https://mlsanity.com/paper/2610.15711/) (2026)

## Abstract (arXiv)

The genome sequence contains the blueprint for governing cellular processes. While the availability of genomes has vastly increased over the last decades, experimental annotation of the various functional, non-coding and regulatory elements encoded in the DNA sequence remains both expensive and challenging. This has sparked interest in unsupervised language modeling of genomic DNA, a paradigm that has seen great success for protein sequence data. Although various DNA language models have been proposed, evaluation tasks often differ between individual works, and might not fully recapitulate the fundamental challenges of genome annotation, including the length, scale and sparsity of the data. In this study, we introduce BEND, a Benchmark for DNA language models, featuring a collection of realistic and biologically meaningful downstream tasks defined on the human genome. We find that embeddings from current DNA LMs can approach performance of expert methods on some tasks, but only capture limited information about long-range features. BEND is available at https://github.com/frederikkemarin/BEND.