mlsanityData to 7 Oct 2026

Enhancing the Protein Tertiary Structure Prediction by Multiple Sequence Alignment Generation

arXiv 2306.01824 ↗First posted 2 Jun 2023q-bio.QM cs.CE cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

The field of protein folding research has been greatly advanced by deep learning methods, with AlphaFold2 (AF2) demonstrating exceptional performance and atomic-level precision. As co-evolution is integral to protein structure prediction, AF2's accuracy is significantly influenced by the depth of multiple sequence alignment (MSA), which requires extensive exploration of a large protein database for similar sequences. However, not all protein sequences possess abundant homologous families, and consequently, AF2's performance can degrade on such queries, at times failing to produce meaningful results. To address this, we introduce a novel generative language model, MSA-Augmenter, which leverages protein-specific attention mechanisms and large-scale MSAs to generate useful, novel protein sequences not currently found in databases. These sequences supplement shallow MSAs, enhancing the accuracy of structural property predictions. Our experiments on CASP14 demonstrate that MSA-Augmenter can generate de novo sequences that retain co-evolutionary information from inferior MSAs, thereby improving protein structure prediction quality on top of strong AF2.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 15 of its 31 references that are in the map. First posted on arXiv 2 Jun 2023.

Builds on

15 of 31 references.

References · 15

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2020Longformer: The Long-Document Transformercited by 3,362
  7. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  8. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
  9. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  10. 2019Axial Attention in Multidimensional Transformerscited by 350
  11. 2020Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processingcited by 161
  12. 2022ProGen2: Exploring the Boundaries of Protein Language Modelscited by 156
3 more
  1. 2020ProGen: Language Modeling for Protein Generationcited by 127
  2. 2020BERTology Meets Biology: Interpreting Attention in Protein Language Modelscited by 85
  3. 2022Generative power of a protein language model trained on multiple sequence alignmentscited by 9

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 3.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2022Protein language models trained on multiple sequence alignments learn phylogenetic relationships
  2. 2026Training-Free Generation of Protein Sequences from Small Family Alignments via Stochastic Attention
  3. 2026ProtAlign: Contrastive learning paradigm for Sequence and structure alignment
  4. 2024Improving Inverse Folding for Peptide Design with Diversity-regularized Direct Preference Optimization
  5. 2025ProtInvTree: Deliberate Protein Inverse Folding with Reward-guided Tree Search
  6. 2025PRISM: Enhancing Protein Inverse Folding through Fine-Grained Retrieval on Structure-Sequence Multimodal Representations
  7. 2022Protein Sequence and Structure Co-Design with Equivariant Translation
  8. 2025Protein Structure Tokenization: Benchmarking and New Recipe