# HelixFold-Single: MSA-free Protein Structure Prediction by Using Protein Language Model as an Alternative

arXiv: https://arxiv.org/abs/2207.13921 · HTML: https://mlsanity.com/paper/2207.13921/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 46 references that are in the map.
- First posted on arXiv 28 Jul 2022.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [On Layer Normalization in the Transformer Architecture](https://mlsanity.com/paper/2002.04745/) (2020)
- [Evaluating Protein Transfer Learning with TAPE](https://mlsanity.com/paper/1906.08230/) (2019)
- [ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computing](https://mlsanity.com/paper/2007.06225/) (2020)
- [Modeling Protein Using Large-scale Pretrain Language Model](https://mlsanity.com/paper/2108.07435/) (2021)
- [HelixFold: An Efficient Implementation of AlphaFold2 using PaddlePaddle](https://mlsanity.com/paper/2207.05477/) (2022)

## Built on by (most cited first, 15 of 15)

- [Large Language Models for Medicine: A Survey](https://mlsanity.com/paper/2405.13055/) (2024)
- [A Survey of Geometric Graph Neural Networks: Data Structures, Models and Applications](https://mlsanity.com/paper/2403.00485/) (2024)
- [Technical Report of HelixFold3 for Biomolecular Structure Prediction](https://mlsanity.com/paper/2408.16975/) (2024)
- [Computational Protein Science in the Era of Large Language Models (LLMs)](https://mlsanity.com/paper/2501.10282/) (2025)
- [Pre-Training on Large-Scale Generated Docking Conformations with HelixDock to Unlock the Potential of Protein-ligand Structure Prediction Models](https://mlsanity.com/paper/2310.13913/) (2023)
- [Unifying Sequences, Structures, and Descriptions for Any-to-Any Protein Generation with the Large Multimodal Model HelixProtX](https://mlsanity.com/paper/2407.09274/) (2024)
- [Large Language Models for Bioinformatics](https://mlsanity.com/paper/2501.06271/) (2025)
- [Advances in Protein Representation Learning: Methods, Applications, and Future Directions](https://mlsanity.com/paper/2503.16659/) (2025)
- [ImmunoLingo: Linguistics-based formalization of the antibody language](https://mlsanity.com/paper/2209.12635/) (2022)
- [HelixFold-Multimer: Elevating Protein Complex Structure Prediction to New Heights](https://mlsanity.com/paper/2404.10260/) (2024)
- [Understanding the Natural Language of DNA using Encoder-Decoder Foundation Models with Byte-level Precision](https://mlsanity.com/paper/2311.02333/) (2023)
- [Boosting Protein Language Models with Negative Sample Mining](https://mlsanity.com/paper/2405.17902/) (2024)
- [AMix-1: A Pathway to Test-Time Scalable Protein Foundation Model](https://mlsanity.com/paper/2507.08920/) (2025)
- [One protein is all you need](https://mlsanity.com/paper/2411.02109/) (2024)
- [Protein Structure Prediction: From Evolutionary Constraints to Generative Modeling](https://mlsanity.com/paper/2608.16094/) (2026)

## Abstract (arXiv)

AI-based protein structure prediction pipelines, such as AlphaFold2, have achieved near-experimental accuracy. These advanced pipelines mainly rely on Multiple Sequence Alignments (MSAs) as inputs to learn the co-evolution information from the homologous sequences. Nonetheless, searching MSAs from protein databases is time-consuming, usually taking dozens of minutes. Consequently, we attempt to explore the limits of fast protein structure prediction by using only primary sequences of proteins. HelixFold-Single is proposed to combine a large-scale protein language model with the superior geometric learning capability of AlphaFold2. Our proposed method, HelixFold-Single, first pre-trains a large-scale protein language model (PLM) with thousands of millions of primary sequences utilizing the self-supervised learning paradigm, which will be used as an alternative to MSAs for learning the co-evolution information. Then, by combining the pre-trained PLM and the essential components of AlphaFold2, we obtain an end-to-end differentiable model to predict the 3D coordinates of atoms from only the primary sequence. HelixFold-Single is validated in datasets CASP14 and CAMEO, achieving competitive accuracy with the MSA-based methods on the targets with large homologous families. Furthermore, HelixFold-Single consumes much less time than the mainstream pipelines for protein structure prediction, demonstrating its potential in tasks requiring many predictions. The code of HelixFold-Single is available at https://github.com/PaddlePaddle/PaddleHelix/tree/dev/apps/protein_folding/helixfold-single, and we also provide stable web services on https://paddlehelix.baidu.com/app/drug/protein-single/forecast.