# Predicting Chemical Properties using Self-Attention Multi-task Learning based on SMILES Representation

arXiv: https://arxiv.org/abs/2010.11272 · HTML: https://mlsanity.com/paper/2010.11272/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Oct 2020.

## Built on by (most cited first, 6 of 6)

- [Geometric Deep Learning on Molecular Representations](https://mlsanity.com/paper/2107.12375/) (2021)
- [A smile is all you need: Predicting limiting activity coefficients from SMILES with natural language processing](https://mlsanity.com/paper/2206.07048/) (2022)
- [SPT-NRTL: A physics-guided machine learning model to predict thermodynamically consistent activity coefficients](https://mlsanity.com/paper/2209.04135/) (2022)
- [Motif-based Graph Representation Learning with Application to Chemical Molecules](https://mlsanity.com/paper/2208.04529/) (2022)
- [Advancing Drug Discovery with Enhanced Chemical Understanding via Asymmetric Contrastive Multimodal Learning](https://mlsanity.com/paper/2311.06456/) (2023)
- [Advancements in Molecular Property Prediction: A Survey of Single and Multimodal Approaches](https://mlsanity.com/paper/2408.09461/) (2024)

## Abstract (arXiv)

In the computational prediction of chemical compound properties, molecular descriptors and fingerprints encoded to low dimensional vectors are used. The selection of proper molecular descriptors and fingerprints is both important and challenging as the performance of such models is highly dependent on descriptors. To overcome this challenge, natural language processing models that utilize simplified molecular input line-entry system as input were studied, and several transformer-variant models achieved superior results when compared with conventional methods. In this study, we explored the structural differences of the transformer-variant model and proposed a new self-attention based model. The representation learning performance of the self-attention module was evaluated in a multi-task learning environment using imbalanced chemical datasets. The experiment results showed that our model achieved competitive outcomes on several benchmark datasets. The source code of our experiment is available at https://github.com/arwhirang/sa-mtl and the dataset is available from the same URL.