# GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models

arXiv: https://arxiv.org/abs/2406.01627 · HTML: https://mlsanity.com/paper/2406.01627/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 33 references that are in the map.
- First posted on arXiv 1 Jun 2024.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation](https://mlsanity.com/paper/2108.12409/) (2021)
- [Long Range Arena: A Benchmark for Efficient Transformers](https://mlsanity.com/paper/2011.04006/) (2020)
- [Hyena Hierarchy: Towards Larger Convolutional Language Models](https://mlsanity.com/paper/2302.10866/) (2023)
- [HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution](https://mlsanity.com/paper/2306.15794/) (2023)
- [DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genome](https://mlsanity.com/paper/2306.15006/) (2023)
- [Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modeling](https://mlsanity.com/paper/2403.03234/) (2024)
- [BEND: Benchmarking DNA Language Models on biologically meaningful tasks](https://mlsanity.com/paper/2311.12570/) (2023)

## Built on by (most cited first, 10 of 10)

- [MoE-Enhanced Explainable Deep Manifold Transformation for Complex Data Embedding and Visualization](https://mlsanity.com/paper/2410.19504/) (2024)
- [Life-Code: Central Dogma Modeling with Multi-Omics Sequence Unification](https://mlsanity.com/paper/2502.07299/) (2025)
- [Gene42: Long-Range Genomic Foundation Model With Dense Attention](https://mlsanity.com/paper/2503.16565/) (2025)
- [MergeDNA: Context-aware Genome Modeling with Dynamic Tokenization through Token Merging](https://mlsanity.com/paper/2511.14806/) (2025)
- [dnaGrinder: a lightweight and high-capacity genomic foundation model](https://mlsanity.com/paper/2409.15697/) (2024)
- [Does your model understand genes? A benchmark of gene properties for biological and text models](https://mlsanity.com/paper/2412.04075/) (2024)
- [Embedding Is (Almost) All You Need: Retrieval-Augmented Inference for Generalizable Genomic Prediction Tasks](https://mlsanity.com/paper/2508.04757/) (2025)
- [NABench: Large-Scale Benchmarks of Nucleotide Foundation Models for Fitness Prediction](https://mlsanity.com/paper/2511.02888/) (2025)
- [OmniGenBench: Automating Large-scale in-silico Benchmarking for Genomic Foundation Models](https://mlsanity.com/paper/2410.01784/) (2024)
- [Event Fields: Learning Latent Event Structure for Waveform Foundation Models](https://mlsanity.com/paper/2605.08685/) (2026)

## Abstract (arXiv)

The Genomic Foundation Model (GFM) paradigm is expected to facilitate the extraction of generalizable representations from massive genomic data, thereby enabling their application across a spectrum of downstream applications. Despite advancements, a lack of evaluation framework makes it difficult to ensure equitable assessment due to experimental settings, model intricacy, benchmark datasets, and reproducibility challenges. In the absence of standardization, comparative analyses risk becoming biased and unreliable. To surmount this impasse, we introduce GenBench, a comprehensive benchmarking suite specifically tailored for evaluating the efficacy of Genomic Foundation Models. GenBench offers a modular and expandable framework that encapsulates a variety of state-of-the-art methodologies. Through systematic evaluations of datasets spanning diverse biological domains with a particular emphasis on both short-range and long-range genomic tasks, firstly including the three most important DNA tasks covering Coding Region, Non-Coding Region, Genome Structure, etc. Moreover, We provide a nuanced analysis of the interplay between model architecture and dataset characteristics on task-specific performance. Our findings reveal an interesting observation: independent of the number of parameters, the discernible difference in preference between the attention-based and convolution-based models on short- and long-range tasks may provide insights into the future design of GFM.