LA4SR: illuminating the dark proteome with generative AI
Not on the map. Why?
Abstract (arXiv)
AI language models (LMs) show promise for biological sequence analysis. We re-engineered open-source LMs (GPT-2, BLOOM, DistilRoBERTa, ELECTRA, and Mamba, ranging from 70M to 12B parameters) for microbial sequence classification. The models achieved F1 scores up to 95 and operated 16,580x faster and at 2.9x the recall of BLASTP. They effectively classified the algal dark proteome - uncharacterized proteins comprising about 65% of total proteins - validated on new data including a new, complete Hi-C/Pacbio Chlamydomonas genome. Larger (>1B) LA4SR models reached high accuracy (F1 > 86) when trained on less than 2% of available data, rapidly achieving strong generalization capacity. High accuracy was achieved when training data had intact or scrambled terminal information, demonstrating robust generalization to incomplete sequences. Finally, we provide custom AI explainability software tools for attributing amino acid patterns to AI generative processes and interpret their outputs in evolutionary and biophysical contexts.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 68 references that are in the map. First posted on arXiv 11 Nov 2024.
Builds on
22 of 68 references.
References · 22
- 2017Attention Is All You Needcited by 60,361
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
- 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
- 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
- 2017Axiomatic Attribution for Deep Networkscited by 3,384
- 2023Mistral 7Bcited by 3,164
- 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
- 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
- 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
10 more
- 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,463
- 2017Learning Important Features Through Propagating Activation Differencescited by 1,458
- 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
- 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
- 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 713
- 2013Speech Recognition with Deep Recurrent Neural Networkscited by 669
- 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
- 2020Captum: A unified and generic model interpretability library for PyTorchcited by 547
- 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 509
- 2018Measuring the Effects of Data Parallelism on Neural Network Trainingcited by 193
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026QLoRA Fine-Tuning of Ministral LLM for Sequence-to-Function Protein Annotation
- 2025Lyra: An Efficient and Expressive Subquadratic Architecture for Modeling Biological Sequences
- 2023PoET: A generative model of protein families as sequences-of-sequences
- 2024Bio-xLSTM: Generative modeling, representation and in-context learning of biological and chemical sequences
- 2025Steering Protein Language Models
- 2024TourSynbio: A Multi-Modal Large Model and Agent Framework to Bridge Text and Protein Sequences for Protein Engineering
- 2022Protein language models trained on multiple sequence alignments learn phylogenetic relationships
- 2026The limits of bio-molecular modeling with large language models : a cross-scale evaluation