# Benchmarking Accuracy and Generalizability of Four Graph Neural Networks Using Large In Vitro ADME Datasets from Different Chemical Spaces

arXiv: https://arxiv.org/abs/2111.13964 · HTML: https://mlsanity.com/paper/2111.13964/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 28 references that are in the map.
- First posted on arXiv 27 Nov 2021.

## Builds on (most cited first, 8 of 8)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Molecular Graph Convolutions: Moving Beyond Fingerprints](https://mlsanity.com/paper/1603.00856/) (2016)
- [Massively Multitask Networks for Drug Discovery](https://mlsanity.com/paper/1502.02072/) (2015)
- [DGL-LifeSci: An Open-Source Toolkit for Deep Learning on Graphs in Life Science](https://mlsanity.com/paper/2106.14232/) (2021)
- [Modeling Industrial ADMET Data with Multitask Networks](https://mlsanity.com/paper/1606.08793/) (2016)
- [Step Change Improvement in ADMET Prediction with PotentialNet Deep Featurization](https://mlsanity.com/paper/1903.11789/) (2019)

## Built on by (most cited first, 5 of 5)

- [Molecule-Morphology Contrastive Pretraining for Transferable Molecular Representation](https://mlsanity.com/paper/2305.09790/) (2023)
- [mCLM: A Modular Chemical Language Model that Generates Functional and Makeable Molecules](https://mlsanity.com/paper/2505.12565/) (2025)
- [Machine Learning Small Molecule Properties in Drug Discovery](https://mlsanity.com/paper/2308.12354/) (2023)
- [Pretraining Graph Transformers with Atom-in-a-Molecule Quantum Properties for Improved ADMET Modeling](https://mlsanity.com/paper/2410.08024/) (2024)
- [PlayMolecule pKAce: Small Molecule Protonation through Equivariant Neural Networks](https://mlsanity.com/paper/2407.11103/) (2024)

## Abstract (arXiv)

In this work, we benchmark a variety of single- and multi-task graph neural network (GNN) models against lower-bar and higher-bar traditional machine learning approaches employing human engineered molecular features. We consider four GNN variants -- Graph Convolutional Network (GCN), Graph Attention Network (GAT), Message Passing Neural Network (MPNN), and Attentive Fingerprint (AttentiveFP). So far deep learning models have been primarily benchmarked using lower-bar traditional models solely based on fingerprints, while more realistic benchmarks employing fingerprints, whole-molecule descriptors and predictions from other related endpoints (e.g., LogD7.4) appear to be scarce for industrial ADME datasets. In addition to time-split test sets based on Genentech data, this study benefits from the availability of measurements from an external chemical space (Roche data). We identify GAT as a promising approach to implementing deep learning models. While all GNN models significantly outperform lower-bar benchmark traditional models solely based on fingerprints, only GATs seem to offer a small but consistent improvement over higher-bar benchmark traditional models. Finally, the accuracy of in vitro assays from different laboratories predicting the same experimental endpoints appears to be comparable with the accuracy of GAT single-task models, suggesting that most of the observed error from the models is a function of the experimental error propagation.