MI-PEFT: Mixture-of-Experts Integrated Parameter-Efficient Fine-Tuning Protein Language Models Improves Acidophilic Proteins Classification
On the map: Mixture-of-experts → Medical and biomedical MoE
Abstract (arXiv)
Acidophilic proteins that remain stable and functional under highly acidic conditions, are important for industrial biocatalysis, acid-related bioprocessing, and the discovery of acid-stable enzymes. However, their identification relies heavily on time-consuming experimental screening methods. With the rapid growth of protein sequence databases, the need for computational identification methods that are both accurate and efficient has become stronger. The emergence of protein language models (PLMs) has significantly improved the sequence representation of downstream biological prediction tasks. This paper proposes MI-PEFT, a mixture-of-experts integrated parameter-efficient fine-tuning framework. Built on the ESM C-600M backbone, the framework incorporates LoRA-based PEFT methods and a DeepSeekMoE-based classification head to resolve the limitations of PEFT and significantly improve computational efficiency. Notably, this task is characterized by a significant class imbalance in the dataset, making high specificity particularly challenging. The experimental results demonstrate that MI-PEFT on PLMs, especially C3A, serves as an efficient tool for identifying acidophilic proteins and a constrained pathway that helps resolve class-imbalance by preserving the pretrained representations.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Medical and biomedical MoE, one of 12 directions in Mixture-of-experts. Builds on 14 of its 37 references that are in the map. First posted on arXiv 7 Sep 2026.
Builds on
14 of 37 references.
References · 14
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
- 2017On Calibration of Modern Neural Networkscited by 4,860
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
- 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
- 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
- 2019Evaluating Protein Transfer Learning with TAPEcited by 229
- 2023Controlling Text-to-Image Diffusion by Orthogonal Finetuningcited by 207
- 2025RandLoRA: Full-rank parameter-efficient fine-tuning of large modelscited by 34
- 2024Sparse High Rank Adapterscited by 17
- 2024ESM All-Atom: Multi-scale Protein Language Model for Unified Molecular Modelingcited by 7
2 more
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Structure-Enhanced Protein Instruction Tuning: Towards General-Purpose Protein Understanding with LLMs
- 2025Beyond Protein Language Models: An Agentic LLM Framework for Mechanistic Enzyme Design
- 2024X-LoRA: Mixture of Low-Rank Adapter Experts, a Flexible Framework for Large Language Models with Applications in Protein Mechanics and Molecular Design
- 2026Multimodal Protein Language Models for Enzyme Kinetic Parameters: From Substrate Recognition to Conformational Adaptation
- 2026The limits of bio-molecular modeling with large language models : a cross-scale evaluation
- 2024MolecularGPT: Open Large Language Model (LLM) for Few-Shot Molecular Property Prediction
- 2026Knowledge before Reasoning: EC-Reason-Bench, a Training-Free Diagnostic Benchmark for LLM Enzyme Classification
- 2026Protein Language Models Diverge from Natural Language: Comparative Analysis and Improved Inference