Do Vision CIL Methods Transfer to Protein? A Diagnostic Benchmark of Frozen Protein Encoders
Not on the map. Why?
Abstract (arXiv)
Protein function annotation operates over an expanding label space, making class-incremental learning (CIL) a natural framework for incorporating newly discovered functions without full retraining. Yet existing CIL methods have been developed largely in vision and have not been systematically evaluated over frozen protein representations. We conduct a controlled benchmark of nine representative CIL methods across five sequence-, structure-, and sequence--structure-based protein encoders and two enzyme function datasets at different scales, while keeping the encoders frozen and evaluation settings consistent across representations. We find that no method transfers reliably across encoders, and that these failures persist across alternative phase constructions. Diagnostic analyses reveal that this instability arises from weak gradient-based stability signals, substantial heterogeneity in embedding geometry, and covariance assumptions whose appropriate capacity changes with both dataset scale and encoder. These findings suggest that the representation itself should be treated as a central component of continual-learning design rather than a passive feature extractor. More generally, reliable protein CIL will require encoder-aware methods that adapt their statistical assumptions and capacity to the geometry of the representation on which they operate. Code is available at https://github.com/zygao930/ProteinCIL.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 39 references that are in the map. First posted on arXiv 1 Oct 2026.
Builds on
20 of 39 references.
References · 20
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
- 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
- 2017Gradient Episodic Memory for Continual Learningcited by 1,898
- 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 911
- 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 742
- 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 642
- 2022CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learningcited by 450
- 2020Class-incremental learning: survey and performance evaluation on image classificationcited by 449
- 2019Evaluating Protein Transfer Learning with TAPEcited by 231
- 2023Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Needcited by 184
- 2023SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Modelcited by 180
- 2023RanPAC: Random Projections and Pre-trained Models for Continual Learningcited by 164
8 more
- 2022Protein Representation Learning by Geometric Structure Pretrainingcited by 162
- 2024Continual Learning with Pre-Trained Models: A Surveycited by 154
- 2022FeTrIL: Feature Translation for Exemplar-Free Class-Incremental Learningcited by 121
- 2024Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learningcited by 117
- 2023FeCAM: Exploiting the Heterogeneity of Class Distributions in Exemplar-Free Continual Learningcited by 87
- 2024MOS: Model Surgery for Pre-Trained Model-Based Class-Incremental Learningcited by 36
- 2021Relational Experience Replay: Continual Learning by Adaptively Tuning Task-wise Relationshipcited by 12
- 2024Endowing Protein Language Models with Structural Knowledgecited by 11
Built on by
Not cited yet.