# Prot2Text-V2: Protein Function Prediction with Multimodal Contrastive Alignment

arXiv: https://arxiv.org/abs/2505.11194 · HTML: https://mlsanity.com/paper/2505.11194/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 May 2025.

## Built on by (most cited first, 7 of 7)

- [Cell2Text: Multimodal LLM for Generating Single-Cell Descriptions from RNA-Seq Data](https://mlsanity.com/paper/2509.24840/) (2025)
- [Interleaved Tool-Call Reasoning for Protein Function Understanding](https://mlsanity.com/paper/2601.03604/) (2026)
- [Scaling-Aware Adapter for Structure-Grounded LLM Reasoning](https://mlsanity.com/paper/2602.02780/) (2026)
- [ES-Merging: Biological MLLM Merging via Embedding Space Signals](https://mlsanity.com/paper/2603.14405/) (2026)
- [From Snapshots to Symphonies: The Evolution of Protein Prediction from Static Structures to Generative Dynamics and Multimodal Interactions](https://mlsanity.com/paper/2603.18505/) (2026)
- [Unlocking Biological Workflows for Robust Protein-Text Question Answering: A Dual-Dimensional RAG Framework](https://mlsanity.com/paper/2605.17261/) (2026)
- [QLoRA Fine-Tuning of Ministral LLM for Sequence-to-Function Protein Annotation](https://mlsanity.com/paper/2609.24538/) (2026)

## Abstract (arXiv)

Predicting protein function from sequence is a central challenge in computational biology. While existing methods rely heavily on structured ontologies or similarity-based techniques, they often lack the flexibility to express structure-free functional descriptions and novel biological functions. In this work, we introduce Prot2Text-V2, a novel multimodal sequence-to-text model that generates free-form natural language descriptions of protein function directly from amino acid sequences. Our method combines a protein language model as a sequence encoder (ESM-3B) and a decoder-only language model (LLaMA-3.1-8B-Instruct) through a lightweight nonlinear modality projector. A key innovation is our Hybrid Sequence-level Contrastive Alignment Learning (H-SCALE), which improves cross-modal learning by matching mean- and std-pooled protein embeddings with text representations via contrastive loss. After the alignment phase, we apply instruction-based fine-tuning using LoRA on the decoder to teach the model how to generate accurate protein function descriptions conditioned on the protein sequence. We train Prot2Text-V2 on about 250K curated entries from SwissProt and evaluate it under low-homology conditions, where test sequences have low similarity with training samples. Prot2Text-V2 consistently outperforms traditional and LLM-based baselines across various metrics.