# Benchmarking Machine Learning Robustness in Covid-19 Genome Sequence Classification

arXiv: https://arxiv.org/abs/2207.08898 · HTML: https://mlsanity.com/paper/2207.08898/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 65 references that are in the map.
- First posted on arXiv 18 Jul 2022.

## Builds on (most cited first, 13 of 13)

- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Unsupervised Deep Embedding for Clustering Analysis](https://mlsanity.com/paper/1511.06335/) (2015)
- [Uncertainty in Gradient Boosting via Ensembles](https://mlsanity.com/paper/2006.10562/) (2020)
- [Spike2Vec: An Efficient and Scalable Embedding Approach for COVID-19 Spike Sequences](https://mlsanity.com/paper/2109.05019/) (2021)
- [Differentiable sampling of molecular geometries with uncertainty-based adversarial attacks](https://mlsanity.com/paper/2101.11588/) (2021)
- [A k-mer Based Approach for SARS-CoV-2 Variant Identification](https://mlsanity.com/paper/2108.03465/) (2021)
- [PWM2Vec: An Efficient Embedding Approach for Viral Host Specification from Coronavirus Spike Sequences](https://mlsanity.com/paper/2201.02273/) (2022)
- [Efficient Approximation Algorithms for String Kernel Based Sequence Classification](https://mlsanity.com/paper/1712.04264/) (2017)
- [Effective and scalable clustering of SARS-CoV-2 sequences](https://mlsanity.com/paper/2108.08143/) (2021)
- [Protein Folding Neural Networks Are Not Robust](https://mlsanity.com/paper/2109.04460/) (2021)
- [ProDOMA: improve PROtein DOMAin classification for third-generation sequencing reads using deep learning](https://mlsanity.com/paper/2009.12591/) (2020)

## Built on by (most cited first, 6 of 6)

- [Exploring the Physical World Adversarial Robustness of Vehicle Detection](https://mlsanity.com/paper/2308.03476/) (2023)
- [Evaluating COVID-19 Sequence Data Using Nearest-Neighbors Based Network Model](https://mlsanity.com/paper/2211.10546/) (2022)
- [BioSequence2Vec: Efficient Embedding Generation For Biological Sequences](https://mlsanity.com/paper/2304.00291/) (2023)
- [ViralVectors: Compact and Scalable Alignment-free Virome Feature Generation](https://mlsanity.com/paper/2304.02891/) (2023)
- [PCD2Vec: A Poisson Correction Distance-Based Approach for Viral Host Classification](https://mlsanity.com/paper/2304.06731/) (2023)
- [Virus2Vec: Viral Sequence Classification Using Machine Learning](https://mlsanity.com/paper/2304.12328/) (2023)

## Abstract (arXiv)

The rapid spread of the COVID-19 pandemic has resulted in an unprecedented amount of sequence data of the SARS-CoV-2 genome -- millions of sequences and counting. This amount of data, while being orders of magnitude beyond the capacity of traditional approaches to understanding the diversity, dynamics, and evolution of viruses is nonetheless a rich resource for machine learning (ML) approaches as alternatives for extracting such important information from these data. It is of hence utmost importance to design a framework for testing and benchmarking the robustness of these ML models. This paper makes the first effort (to our knowledge) to benchmark the robustness of ML models by simulating biological sequences with errors. In this paper, we introduce several ways to perturb SARS-CoV-2 genome sequences to mimic the error profiles of common sequencing platforms such as Illumina and PacBio. We show from experiments on a wide array of ML models that some simulation-based approaches are more robust (and accurate) than others for specific embedding methods to certain adversarial attacks to the input sequences. Our benchmarking framework may assist researchers in properly assessing different ML models and help them understand the behavior of the SARS-CoV-2 virus or avoid possible future pandemics.