# Protein Folding Neural Networks Are Not Robust

arXiv: https://arxiv.org/abs/2109.04460 · HTML: https://mlsanity.com/paper/2109.04460/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 23 references that are in the map.
- First posted on arXiv 9 Sep 2021.

## Builds on (most cited first, 4 of 4)

- [Square Attack: a query-efficient black-box adversarial attack via random search](https://mlsanity.com/paper/1912.00049/) (2019)
- [RobustBench: a standardized adversarial robustness benchmark](https://mlsanity.com/paper/2010.09670/) (2020)
- [Mind the box: l1-APGD for sparse adversarial attacks on image classifiers](https://mlsanity.com/paper/2103.01208/) (2021)
- [Improving Query Efficiency of Black-box Adversarial Attack](https://mlsanity.com/paper/2009.11508/) (2020)

## Built on by (most cited first, 7 of 7)

- [Data Augmentation for Image Classification using Generative AI](https://mlsanity.com/paper/2409.00547/) (2024)
- [Benchmarking Machine Learning Robustness in Covid-19 Genome Sequence Classification](https://mlsanity.com/paper/2207.08898/) (2022)
- [Neuro Symbolic Reasoning for Planning: Counterexample Guided Inductive Synthesis using Large Language Models and Satisfiability Solving](https://mlsanity.com/paper/2309.16436/) (2023)
- [On the Robustness of AlphaFold: A COVID-19 Case Study](https://mlsanity.com/paper/2301.04093/) (2023)
- [Neural Stochastic Differential Equations for Robust and Explainable Analysis of Electromagnetic Unintended Radiated Emissions](https://mlsanity.com/paper/2309.15386/) (2023)
- [Anti-virus Autobots: Predicting More Infectious Virus Variants for Pandemic Prevention through Deep Learning](https://mlsanity.com/paper/2205.14854/) (2022)
- [AF2-Mutation: Adversarial Sequence Mutations against AlphaFold2 on Protein Tertiary Structure Prediction](https://mlsanity.com/paper/2305.08929/) (2023)

## Abstract (arXiv)

Deep neural networks such as AlphaFold and RoseTTAFold predict remarkably accurate structures of proteins compared to other algorithmic approaches. It is known that biologically small perturbations in the protein sequence do not lead to drastic changes in the protein structure. In this paper, we demonstrate that RoseTTAFold does not exhibit such a robustness despite its high accuracy, and biologically small perturbations for some input sequences result in radically different predicted protein structures. This raises the challenge of detecting when these predicted protein structures cannot be trusted. We define the robustness measure for the predicted structure of a protein sequence to be the inverse of the root-mean-square distance (RMSD) in the predicted structure and the structure of its adversarially perturbed sequence. We use adversarial attack methods to create adversarial protein sequences, and show that the RMSD in the predicted protein structure ranges from 0.119rA to 34.162rA when the adversarial perturbations are bounded by 20 units in the BLOSUM62 distance. This demonstrates very high variance in the robustness measure of the predicted structures. We show that the magnitude of the correlation (0.917) between our robustness measure and the RMSD between the predicted structure and the ground truth is high, that is, the predictions with low robustness measure cannot be trusted. This is the first paper demonstrating the susceptibility of RoseTTAFold to adversarial attacks.