mlsanityData to 7 Oct 2026

ImmunoLingo: Linguistics-based formalization of the antibody language

arXiv 2209.12635 ↗First posted 26 Sep 2022q-bio.QM cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Apparent parallels between natural language and biological sequence have led to a recent surge in the application of deep language models (LMs) to the analysis of antibody and other biological sequences. However, a lack of a rigorous linguistic formalization of biological sequence languages, which would define basic components, such as lexicon (i.e., the discrete units of the language) and grammar (i.e., the rules that link sequence well-formedness, structure, and meaning) has led to largely domain-unspecific applications of LMs, which do not take into account the underlying structure of the biological sequences studied. A linguistic formalization, on the other hand, establishes linguistically-informed and thus domain-adapted components for LM applications. It would facilitate a better understanding of how differences and similarities between natural language and biological sequences influence the quality of LMs, which is crucial for the design of interpretable models with extractable sequence-functions relationship rules, such as the ones underlying the antibody specificity prediction problem. Deciphering the rules of antibody specificity is crucial to accelerating rational and in silico biotherapeutic drug design. Here, we formalize the properties of the antibody language and thereby establish not only a foundation for the application of linguistic tools in adaptive immune receptor analysis but also for the systematic immunolinguistic studies of immune receptor specificity in general.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 35 of its 178 references that are in the map. First posted on arXiv 26 Sep 2022.

Builds on

35 of 178 references.

References · 35

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2018Deep contextualized word representationscited by 2,285
  4. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  5. 2020A Primer in BERTology: What we know about how BERT workscited by 807
  6. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
  7. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  8. 2016Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependenciescited by 350
  9. 2019Evaluating Protein Transfer Learning with TAPEcited by 229
  10. 2022Neural Networks and the Chomsky Hierarchycited by 214
  11. 2019Assessing BERT's Syntactic Abilitiescited by 191
  12. 2020ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computingcited by 162
23 more
  1. 2022ProGen2: Exploring the Boundaries of Protein Language Modelscited by 156
  2. 2021Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLPcited by 130
  3. 2022Pretrained Language Models for Text Generation: A Surveycited by 121
  4. 2019Machine learning for molecular simulationcited by 109
  5. 2018Diachronic word embeddings and semantic shifts: a surveycited by 90
  6. 2020BERTology Meets Biology: Interpreting Attention in Protein Language Modelscited by 85
  7. 2022PEER: A Comprehensive and Multi-Task Benchmark for Protein Sequence Understandingcited by 77
  8. 2020Does syntax need to grow on trees? Sources of hierarchical inductive bias in sequence-to-sequence networkscited by 72
  9. 2021Keeping it Simple: Language Models can learn Complex Molecular Distributionscited by 68
  10. 2017Extracting Automata from Recurrent Neural Networks Using Queries and Counterexamplescited by 66
  11. 2020TorchMD: A deep learning framework for molecular simulationscited by 55
  12. 2021Deciphering antibody affinity maturation with language models and weakly supervised learningcited by 52
  13. 2022ID and OOD Performance Are Sometimes Inversely Correlated on Real-world Datasetscited by 44
  14. 2019Tree-Transformer: A Transformer-Based Method for Correction of Tree-Structured Datacited by 20
  15. 2022HelixFold-Single: MSA-free Protein Structure Prediction by Using Protein Language Model as an Alternativecited by 15
  16. 2022Slangvolution: A Causal Analysis of Semantic Change and Frequency Dynamics in Slangcited by 14
  17. 2020Morphological Word Segmentation on Agglutinative Languages for Neural Machine Translationcited by 11
  18. 2017Model Transfer for Tagging Low-resource Languages using a Bilingual Dictionarycited by 10
  19. 2021Neural message passing for joint paratope-epitope predictioncited by 10
  20. 2020Neural Polysynthetic Language Modellingcited by 7
  21. 2022Linguistically inspired roadmap for building biologically reliable protein language modelscited by 7
  22. 2019To lemmatize or not to lemmatize: how word normalisation affects ELMo performance in word sense disambiguationcited by 4
  23. 2022Transformer with Tree-order Encoding for Neural Program Generationcited by 3

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 0; 2024: 0; 2025: 1.

2 papers.

  1. 2022Incorporating Pre-training Paradigm for Antibody Sequence-Structure Co-designcited by 3
  2. 2025Attribution assignment for deep-generative sequence models enables interpretability analysis using positive-only datacited by 0