# DART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA

arXiv: https://arxiv.org/abs/2412.05430 · HTML: https://mlsanity.com/paper/2412.05430/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 55 references that are in the map.
- First posted on arXiv 6 Dec 2024.

## Builds on (most cited first, 5 of 5)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Large Language Models: A Survey](https://mlsanity.com/paper/2402.06196/) (2024)
- [DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genome](https://mlsanity.com/paper/2306.15006/) (2023)
- [Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modeling](https://mlsanity.com/paper/2403.03234/) (2024)
- [BEND: Benchmarking DNA Language Models on biologically meaningful tasks](https://mlsanity.com/paper/2311.12570/) (2023)

## Built on by (most cited first, 6 of 6)

- [STRAND: Sequence-Conditioned Transport for Single-Cell Perturbations](https://mlsanity.com/paper/2602.10156/) (2026)
- [BMFM-DNA: A SNP-aware DNA foundation model to capture variant effects](https://mlsanity.com/paper/2507.05265/) (2025)
- [NABench: Large-Scale Benchmarks of Nucleotide Foundation Models for Fitness Prediction](https://mlsanity.com/paper/2511.02888/) (2025)
- [GenomeQA: Benchmarking General Large Language Models for Genome Sequence Understanding](https://mlsanity.com/paper/2604.05774/) (2026)
- [Calibrating Generative Models to Feature Distributions with MMD Finetuning](https://mlsanity.com/paper/2606.19496/) (2026)
- [Recovery Rates Are Not Comparable Across Transcription Factors: Chance Correction for Attribution Evaluation](https://mlsanity.com/paper/2609.16271/) (2026)

## Abstract (arXiv)

Recent advances in self-supervised models for natural language, vision, and protein sequences have inspired the development of large genomic DNA language models (DNALMs). These models aim to learn generalizable representations of diverse DNA elements, potentially enabling various genomic prediction, interpretation and design tasks. Despite their potential, existing benchmarks do not adequately assess the capabilities of DNALMs on key downstream applications involving an important class of non-coding DNA elements critical for regulating gene activity. In this study, we introduce DART-Eval, a suite of representative benchmarks specifically focused on regulatory DNA to evaluate model performance across zero-shot, probed, and fine-tuned scenarios against contemporary ab initio models as baselines. Our benchmarks target biologically meaningful downstream tasks such as functional sequence feature discovery, predicting cell-type specific regulatory activity, and counterfactual prediction of the impacts of genetic variants. We find that current DNALMs exhibit inconsistent performance and do not offer compelling gains over alternative baseline models for most tasks, while requiring significantly more computational resources. We discuss potentially promising modeling, data curation, and evaluation strategies for the next generation of DNALMs. Our code is available at https://github.com/kundajelab/DART-Eval.