# SpanSeq: Similarity-based sequence data splitting method for improved development and assessment of deep learning projects

arXiv: https://arxiv.org/abs/2402.14482 · HTML: https://mlsanity.com/paper/2402.14482/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 67 references that are in the map.
- First posted on arXiv 22 Feb 2024.

## Builds on (most cited first, 13 of 13)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [A Closer Look at Memorization in Deep Networks](https://mlsanity.com/paper/1706.05394/) (2017)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [Quantifying Memorization Across Neural Language Models](https://mlsanity.com/paper/2202.07646/) (2022)
- [Deduplicating Training Data Makes Language Models Better](https://mlsanity.com/paper/2107.06499/) (2021)
- [What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation](https://mlsanity.com/paper/2008.03703/) (2020)
- [Does Learning Require Memorization? A Short Tale about a Long Tail](https://mlsanity.com/paper/1906.05271/) (2019)
- [Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models](https://mlsanity.com/paper/2205.10770/) (2022)
- [The Adverse Effects of Code Duplication in Machine Learning Models of Code](https://mlsanity.com/paper/1812.06469/) (2018)
- [We Need to Talk About Random Splits](https://mlsanity.com/paper/2005.00636/) (2020)
- [Memorization vs. Generalization: Quantifying Data Leakage in NLP Performance Evaluation](https://mlsanity.com/paper/2102.01818/) (2021)
- [Identity Crisis: Memorization and Generalization under Extreme Overparameterization](https://mlsanity.com/paper/1902.04698/) (2019)
- [Inflation of test accuracy due to data leakage in deep learning-based classification of OCT images](https://mlsanity.com/paper/2202.12267/) (2022)

## Built on by (most cited first, 5 of 5)

- [Evaluation of Seismic Artificial Intelligence with Uncertainty](https://mlsanity.com/paper/2501.14809/) (2025)
- [De novo generation of functional terpene synthases using TpsGPT](https://mlsanity.com/paper/2512.08772/) (2025)
- [ViroBench: Benchmarking Nucleotide Foundation Models on Viral Genomics Tasks](https://mlsanity.com/paper/2605.25388/) (2026)
- [Refnd: Preventing Data Leakage in Relational Datasets](https://mlsanity.com/paper/2607.19376/) (2026)
- [ReLaG: A Scalable Framework Generalizing Random Splits to Data with Latent Relations](https://mlsanity.com/paper/2609.38538/) (2026)

## Abstract (arXiv)

The use of deep learning models in computational biology has increased massively in recent years, and it is expected to continue with the current advances in the fields such as Natural Language Processing. These models, although able to draw complex relations between input and target, are also inclined to learn noisy deviations from the pool of data used during their development. In order to assess their performance on unseen data (their capacity to generalize), it is common to split the available data randomly into development (train/validation) and test sets. This procedure, although standard, has been shown to produce dubious assessments of generalization due to the existing similarity between samples in the databases used. In this work, we present SpanSeq, a database partition method for machine learning that can scale to most biological sequences (genes, proteins and genomes) in order to avoid data leakage between sets. We also explore the effect of not restraining similarity between sets by reproducing the development of two state-of-the-art models on bioinformatics, not only confirming the consequences of randomly splitting databases on the model assessment, but expanding those repercussions to the model development. SpanSeq is available at https://github.com/genomicepidemiology/SpanSeq.