# How Private Are DNA Embeddings? Inverting Foundation Model Representations of Genomic Sequences

arXiv: https://arxiv.org/abs/2603.06950 · HTML: https://mlsanity.com/paper/2603.06950/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 28 references that are in the map.
- First posted on arXiv 6 Mar 2026.

## Builds on (most cited first, 14 of 14)

- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT](https://mlsanity.com/paper/2302.09419/) (2023)
- [A Cookbook of Self-Supervised Learning](https://mlsanity.com/paper/2304.12210/) (2023)
- [The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks](https://mlsanity.com/paper/1911.07135/) (2019)
- [Byte Pair Encoding is Suboptimal for Language Model Pretraining](https://mlsanity.com/paper/2004.03720/) (2020)
- [DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genome](https://mlsanity.com/paper/2306.15006/) (2023)
- [Re-thinking Model Inversion Attacks Against Deep Neural Networks](https://mlsanity.com/paper/2304.01669/) (2023)
- [Language Models are Injective and Hence Invertible](https://mlsanity.com/paper/2510.15511/) (2025)
- [Learning to Invert: Simple Adaptive Attacks for Gradient Inversion in Federated Learning](https://mlsanity.com/paper/2210.10880/) (2022)
- [BEND: Benchmarking DNA Language Models on biologically meaningful tasks](https://mlsanity.com/paper/2311.12570/) (2023)
- [Boosting Model Inversion Attacks with Adversarial Examples](https://mlsanity.com/paper/2306.13965/) (2023)
- [Robust Representation Learning for Privacy-Preserving Machine Learning: A Multi-Objective Autoencoder Approach](https://mlsanity.com/paper/2309.04427/) (2023)

## Built on by (most cited first, 1 of 1)

- [CipherGenome: Homomorphic Inference for Genomic Mixture-of-Experts](https://mlsanity.com/paper/2609.35883/) (2026)

## Abstract (arXiv)

DNA foundation models have become transformative tools in bioinformatics and healthcare applications. Trained on vast genomic datasets, these models can be used to generate sequence embeddings, dense vector representations that capture complex genomic information. These embeddings are increasingly being shared via Embeddings-as-a-Service (EaaS) frameworks to facilitate downstream tasks, while supposedly protecting the privacy of the underlying raw sequences. However, as this practice becomes more prevalent, the security of these representations is being called into question. This study evaluates the resilience of DNA foundation models to model inversion attacks, whereby adversaries attempt to reconstruct sensitive training data from model outputs. In our study, the model's output for reconstructing the DNA sequence is a zero-shot embedding, which is then fed to a decoder. We evaluated the privacy of three DNA foundation models: DNABERT-2, Evo 2, and Nucleotide Transformer v2 (NTv2). Our results show that per-token embeddings allow near-perfect sequence reconstruction across all models. For mean-pooled embeddings, reconstruction quality degrades as sequence length increases, though it remains substantially above random baselines. Evo 2 and NTv2 prove to be most vulnerable, especially for shorter sequences with reconstruction similarities > 90%, while DNABERT-2's BPE tokenization provides the greatest resilience. We found that the correlation between embedding similarity and sequence similarity was a key predictor of reconstruction success. Our findings emphasize the urgent need for privacy-aware design in genomic foundation models prior to their widespread deployment in EaaS settings. Training code, model weights and evaluation pipeline are released on: https://github.com/not-a-feature/DNA-Embedding-Inversion.