# Protein-Mamba: Biological Mamba Models for Protein Function Prediction

arXiv: https://arxiv.org/abs/2409.14617 · HTML: https://mlsanity.com/paper/2409.14617/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Core SSM architectures for language, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 11 of its 34 references that are in the map.
- First posted on arXiv 22 Sep 2024.

## Placed in

- State-space sequence models (S4, Mamba) / [Core SSM architectures for language](https://mlsanity.com/ssm/core-ssm-architectures-for-language/)

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [An Empirical Study of Mamba-based Language Models](https://mlsanity.com/paper/2406.07887/) (2024)
- [MambaOut: Do We Really Need Mamba for Vision?](https://mlsanity.com/paper/2405.07992/) (2024)
- [PEER: A Comprehensive and Multi-Task Benchmark for Protein Sequence Understanding](https://mlsanity.com/paper/2206.02096/) (2022)
- [Uncertainty Quantification on Clinical Trial Outcome Prediction](https://mlsanity.com/paper/2401.03482/) (2024)
- [SMILES-Mamba: Chemical Mamba Foundation Models for Drug ADMET Prediction](https://mlsanity.com/paper/2408.05696/) (2024)
- [BioMamba: Domain-Adaptive Biomedical Language Models](https://mlsanity.com/paper/2408.02600/) (2024)
- [MambaCapsule: Towards Transparent Cardiac Disease Diagnosis with Electrocardiography Using Mamba Capsule Network](https://mlsanity.com/paper/2407.20893/) (2024)

## Built on by (most cited first, 4 of 4)

- [The Alpha-Alternator: Dynamic Adaptation To Varying Noise Levels In Sequences Using The Vendi Score For Improved Robustness and Performance](https://mlsanity.com/paper/2502.04593/) (2025)
- [Foundation Models for AI-Enabled Biological Design](https://mlsanity.com/paper/2505.11610/) (2025)
- [Alternators With Noise Models](https://mlsanity.com/paper/2505.12544/) (2025)
- [Kinetic-Mamba: Mamba-Assisted Predictions of Stiff Chemical Kinetics](https://mlsanity.com/paper/2512.14471/) (2025)

## Abstract (arXiv)

Protein function prediction is a pivotal task in drug discovery, significantly impacting the development of effective and safe therapeutics. Traditional machine learning models often struggle with the complexity and variability inherent in predicting protein functions, necessitating more sophisticated approaches. In this work, we introduce Protein-Mamba, a novel two-stage model that leverages both self-supervised learning and fine-tuning to improve protein function prediction. The pre-training stage allows the model to capture general chemical structures and relationships from large, unlabeled datasets, while the fine-tuning stage refines these insights using specific labeled datasets, resulting in superior prediction performance. Our extensive experiments demonstrate that Protein-Mamba achieves competitive performance, compared with a couple of state-of-the-art methods across a range of protein function datasets. This model's ability to effectively utilize both unlabeled and labeled data highlights the potential of self-supervised learning in advancing protein function prediction and offers a promising direction for future research in drug discovery.