# dnaHNet: A Scalable and Hierarchical Foundation Model for Genomic Sequence Learning

arXiv: https://arxiv.org/abs/2602.10603 · HTML: https://mlsanity.com/paper/2602.10603/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 32 references that are in the map.
- First posted on arXiv 11 Feb 2026.

## Builds on (most cited first, 14 of 14)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024)
- [Byte Pair Encoding is Suboptimal for Language Model Pretraining](https://mlsanity.com/paper/2004.03720/) (2020)
- [Byte Latent Transformer: Patches Scale Better Than Tokens](https://mlsanity.com/paper/2412.09871/) (2024)
- [DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genome](https://mlsanity.com/paper/2306.15006/) (2023)
- [Dynamic Chunking for End-to-End Hierarchical Sequence Modeling](https://mlsanity.com/paper/2507.07955/) (2025)
- [BioReason: Incentivizing Multimodal Biological Reasoning within a DNA-LLM Model](https://mlsanity.com/paper/2505.23579/) (2025)
- [Model Decides How to Tokenize: Adaptive DNA Sequence Tokenization with MxDNA](https://mlsanity.com/paper/2412.13716/) (2024)
- [MergeDNA: Context-aware Genome Modeling with Dynamic Tokenization through Token Merging](https://mlsanity.com/paper/2511.14806/) (2025)
- [Hybrid Tokenization Strategy for DNA Language Model using Byte Pair Encoding and K-MER Methods](https://mlsanity.com/paper/2507.18570/) (2025)

## Built on by (most cited first, 3 of 3)

- [Self-Play Pretraining with Zero Data](https://mlsanity.com/paper/2609.30063/) (2026)
- [GeneZip: Region-Aware Compression for Long Context DNA Modeling](https://mlsanity.com/paper/2602.17739/) (2026)
- [Byte Language Models: Scaling, Emergent Abstractions, and Information Allocation](https://mlsanity.com/paper/2610.05978/) (2026)

## Abstract (arXiv)

Genomic foundation models have the potential to decode DNA syntax, yet face a fundamental tradeoff in their input representation. Standard fixed-vocabulary tokenizers fragment biologically meaningful motifs such as codons and regulatory elements, while nucleotide-level models preserve biological coherence but incur prohibitive computational costs for long contexts. We introduce dnaHNet, a state-of-the-art tokenizer-free autoregressive model that segments and models genomic sequences end-to-end. Using a differentiable dynamic chunking mechanism, dnaHNet compresses raw nucleotides into latent tokens adaptively, balancing compression with predictive accuracy. Pretrained on prokaryotic genomes, dnaHNet outperforms leading architectures including StripedHyena2 in scaling and efficiency. This recursive chunking yields quadratic FLOP reductions, enabling >3 × inference speedup over Transformers. On zero-shot tasks, dnaHNet achieves superior performance in predicting protein variant fitness and gene essentiality, while automatically discovering hierarchical biological structures without supervision. These results establish dnaHNet as a scalable, interpretable framework for next-generation genomic modeling.