# Multi-Modal CLIP-Informed Protein Editing

arXiv: https://arxiv.org/abs/2407.19296 · HTML: https://mlsanity.com/paper/2407.19296/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 50 references that are in the map.
- First posted on arXiv 27 Jul 2024.

## Builds on (most cited first, 22 of 22)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [ImageBind: One Embedding Space To Bind Them All](https://mlsanity.com/paper/2305.05665/) (2023)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [Evaluating Protein Transfer Learning with TAPE](https://mlsanity.com/paper/1906.08230/) (2019)
- [Vision-Language Pre-Training with Triple Contrastive Learning](https://mlsanity.com/paper/2202.10401/) (2022)
- [ProGen2: Exploring the Boundaries of Protein Language Models](https://mlsanity.com/paper/2206.13517/) (2022)
- [ProtST: Multi-Modality Learning of Protein Sequences and Biomedical Texts](https://mlsanity.com/paper/2301.12040/) (2023)
- [OntoProtein: Protein Pretraining With Gene Ontology Embedding](https://mlsanity.com/paper/2201.11147/) (2022)
- [PRIOR: Prototype Representation Joint Learning from Medical Images and Reports](https://mlsanity.com/paper/2307.12577/) (2023)
- [Revisiting Multimodal Representation in Contrastive Learning: From Patch and Token Embeddings to Finite Discrete Tokens](https://mlsanity.com/paper/2303.14865/) (2023)
- [LMC: Fast Training of GNNs via Subgraph Sampling with Provable Convergence](https://mlsanity.com/paper/2302.00924/) (2023)
- [BioBridge: Bridging Biomedical Foundation Models via Knowledge Graphs](https://mlsanity.com/paper/2310.03320/) (2023)
- [Large scale paired antibody language models](https://mlsanity.com/paper/2403.17889/) (2024)
- [Generative AI for Controllable Protein Sequence Design: A Survey](https://mlsanity.com/paper/2402.10516/) (2024)
- [Label Deconvolution for Node Representation Learning on Large-scale Attributed Graphs against Learning Bias](https://mlsanity.com/paper/2309.14907/) (2023)
- [On Pre-trained Language Models for Antibody](https://mlsanity.com/paper/2301.12112/) (2023)
- [Group-On: Boosting One-Shot Segmentation with Supportive Query](https://mlsanity.com/paper/2404.11871/) (2024)

## Built on by (most cited first, 6 of 6)

- [Bridge-IF: Learning Inverse Protein Folding with Markov Bridges](https://mlsanity.com/paper/2411.02120/) (2024)
- [Computational Protein Science in the Era of Large Language Models (LLMs)](https://mlsanity.com/paper/2501.10282/) (2025)
- [Function-Guided Conditional Generation Using Protein Language Models with Adapters](https://mlsanity.com/paper/2410.03634/) (2024)
- [S2ALM: Sequence-Structure Pre-trained Large Language Model for Comprehensive Antibody Representation Learning](https://mlsanity.com/paper/2411.15215/) (2024)
- [MutaPLM: Protein Language Modeling for Mutation Explanation and Engineering](https://mlsanity.com/paper/2410.22949/) (2024)
- [A Generalist Cross-Domain Molecular Learning Framework for Structure-Based Drug Discovery](https://mlsanity.com/paper/2503.04362/) (2025)

## Abstract (arXiv)

Proteins govern most biological functions essential for life, but achieving controllable protein discovery and optimization remains challenging. Recently, machine learning-assisted protein editing (MLPE) has shown promise in accelerating optimization cycles and reducing experimental workloads. However, current methods struggle with the vast combinatorial space of potential protein edits and cannot explicitly conduct protein editing using biotext instructions, limiting their interactivity with human feedback. To fill these gaps, we propose a novel method called ProtET for efficient CLIP-informed protein editing through multi-modality learning. Our approach comprises two stages: in the pretraining stage, contrastive learning aligns protein-biotext representations encoded by two large language models (LLMs), respectively. Subsequently, during the protein editing stage, the fused features from editing instruction texts and original protein sequences serve as the final editing condition for generating target protein sequences. Comprehensive experiments demonstrated the superiority of ProtET in editing proteins to enhance human-expected functionality across multiple attribute domains, including enzyme catalytic activity, protein stability and antibody specific binding ability. And ProtET improves the state-of-the-art results by a large margin, leading to significant stability improvements of 16.67% and 16.90%. This capability positions ProtET to advance real-world artificial protein editing, potentially addressing unmet academic, industrial, and clinical needs.