# A Hitchhiker's Guide to Deep Chemical Language Processing for Bioactivity Prediction

arXiv: https://arxiv.org/abs/2407.12152 · HTML: https://mlsanity.com/paper/2407.12152/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 58 references that are in the map.
- First posted on arXiv 16 Jul 2024.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Large-Scale Chemical Language Representations Capture Molecular Structure and Properties](https://mlsanity.com/paper/2106.09553/) (2021)
- [DeepDTA: Deep Drug-Target Binding Affinity Prediction](https://mlsanity.com/paper/1801.10193/) (2018)
- [Keeping it Simple: Language Models can learn Complex Molecular Distributions](https://mlsanity.com/paper/2112.03041/) (2021)
- [SMILES Enumeration as Data Augmentation for Neural Network Modeling of Molecules](https://mlsanity.com/paper/1703.07076/) (2017)
- [Gotta be SAFE: A New Framework for Molecular Design](https://mlsanity.com/paper/2310.10773/) (2023)
- [SELFormer: Molecular Representation Learning via SELFIES Language Models](https://mlsanity.com/paper/2304.04662/) (2023)
- [Exploring Chemical Space using Natural Language Processing Methodologies for Drug Discovery](https://mlsanity.com/paper/2002.06053/) (2020)
- [t-SMILES: A Scalable Fragment-based Molecular Representation Framework for De Novo Molecule Generation](https://mlsanity.com/paper/2301.01829/) (2023)
- [Exploiting Pretrained Biochemical Language Models for Targeted Drug Design](https://mlsanity.com/paper/2209.00981/) (2022)
- [Synergy Effect between Convolutional Neural Networks and the Multiplicity of SMILES for Improvement of Molecular Prediction](https://mlsanity.com/paper/1812.04439/) (2018)
- [Structure-based drug discovery with deep learning](https://mlsanity.com/paper/2212.13295/) (2022)

## Abstract (arXiv)

Deep learning has significantly accelerated drug discovery, with 'chemical language' processing (CLP) emerging as a prominent approach. CLP learns from molecular string representations (e.g., Simplified Molecular Input Line Entry Systems [SMILES] and Self-Referencing Embedded Strings [SELFIES]) with methods akin to natural language processing. Despite their growing importance, training predictive CLP models is far from trivial, as it involves many 'bells and whistles'. Here, we analyze the key elements of CLP training, to provide guidelines for newcomers and experts alike. Our study spans three neural network architectures, two string representations, three embedding strategies, across ten bioactivity datasets, for both classification and regression purposes. This 'hitchhiker's guide' not only underscores the importance of certain methodological choices, but it also equips researchers with practical recommendations on ideal choices, e.g., in terms of neural network architectures, molecular representations, and hyperparameter optimization.