# Blind Biological Sequence Denoising with Self-Supervised Set Learning

arXiv: https://arxiv.org/abs/2309.01670 · HTML: https://mlsanity.com/paper/2309.01670/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 67 references that are in the map.
- First posted on arXiv 4 Sep 2023.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [Noise2Noise: Learning Image Restoration without Clean Data](https://mlsanity.com/paper/1803.04189/) (2018)
- [Noise2Self: Blind Denoising by Self-Supervision](https://mlsanity.com/paper/1901.11365/) (2019)
- [High-Quality Self-Supervised Deep Image Denoising](https://mlsanity.com/paper/1901.10277/) (2019)
- [Latent-Variable Non-Autoregressive Neural Machine Translation with Deterministic Inference Using a Delta Posterior](https://mlsanity.com/paper/1908.07181/) (2019)
- [Probabilistic Noise2Void: Unsupervised Content-Aware Denoising](https://mlsanity.com/paper/1906.00651/) (2019)
- [Unsupervised Learning with Stein's Unbiased Risk Estimator](https://mlsanity.com/paper/1805.10531/) (2018)
- [Neural Distance Embeddings for Biological Sequences](https://mlsanity.com/paper/2109.09740/) (2021)
- [Training deep learning based image denoisers from undersampled measurements without ground truth and without image prior](https://mlsanity.com/paper/1806.00961/) (2018)
- [LINDA: Unsupervised Learning to Interpolate in Natural Language Processing](https://mlsanity.com/paper/2112.13969/) (2021)

## Abstract (arXiv)

Biological sequence analysis relies on the ability to denoise the imprecise output of sequencing platforms. We consider a common setting where a short sequence is read out repeatedly using a high-throughput long-read platform to generate multiple subreads, or noisy observations of the same sequence. Denoising these subreads with alignment-based approaches often fails when too few subreads are available or error rates are too high. In this paper, we propose a novel method for blindly denoising sets of sequences without directly observing clean source sequence labels. Our method, Self-Supervised Set Learning (SSSL), gathers subreads together in an embedding space and estimates a single set embedding as the midpoint of the subreads in both the latent and sequence spaces. This set embedding represents the "average" of the subreads and can be decoded into a prediction of the clean sequence. In experiments on simulated long-read DNA data, SSSL methods denoise small reads of ≤ 6 subreads with 17% fewer errors and large reads of >6 subreads with 8% fewer errors compared to the best baseline. On a real dataset of antibody sequences, SSSL improves over baselines on two self-supervised metrics, with a significant improvement on difficult small reads that comprise over 60% of the test set. By accurately denoising these reads, SSSL promises to better realize the potential of high-throughput DNA sequencing data for downstream scientific applications.