mlsanityData to 14 Oct 2026

Why Deep Models Often cannot Beat Non-deep Counterparts on Molecular Property Prediction?

arXiv 2306.17702 ↗First posted 30 Jun 2023cs.LG cs.CEcited by 3

Not on the map. Why?

Abstract (arXiv)

Molecular property prediction (MPP) is a crucial task in the drug discovery pipeline, which has recently gained considerable attention thanks to advances in deep neural networks. However, recent research has revealed that deep models struggle to beat traditional non-deep ones on MPP. In this study, we benchmark 12 representative models (3 non-deep models and 9 deep models) on 14 molecule datasets. Through the most comprehensive study to date, we make the following key observations: (romannumeral 1) Deep models are generally unable to outperform non-deep ones; (romannumeral 2) The failure of deep models on MPP cannot be solely attributed to the small size of molecular datasets. What matters is the irregular molecule data pattern; (romannumeral 3) In particular, tree models using molecular fingerprints as inputs tend to perform better than other competitors. Furthermore, we conduct extensive empirical investigations into the unique patterns of molecule data and inductive biases of various models underlying these phenomena.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 40 of its 83 references that are in the map. First posted on arXiv 30 Jun 2023.

Builds on

40 of 83 references.

References · 40

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  4. 2017Graph Attention Networkscited by 8,923
  5. 2017Inductive Representation Learning on Large Graphscited by 6,320
  6. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  7. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  8. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  9. 2018On the Spectral Bias of Neural Networkscited by 1,341
  10. 2020Graph Contrastive Learning with Augmentationscited by 1,208
  11. 2019Strategies for Pre-training Graph Neural Networkscited by 971
  12. 2021E(n) Equivariant Graph Neural Networkscited by 965
28 more
  1. 2015Convolutional Networks on Graphs for Learning Molecular Fingerprintscited by 754
  2. 2017SchNet: A continuous-filter convolutional neural network for modeling quantum interactionscited by 571
  3. 2020Directional Message Passing for Molecular Graphscited by 561
  4. 2019Analyzing Learned Molecular Representations for Property Predictioncited by 375
  5. 2015Recent Advances in Convolutional Neural Networkscited by 360
  6. 2021GemNet: Universal Directional Graph Neural Networks for Moleculescited by 343
  7. 2021Do Transformers Really Perform Bad for Graph Representation?cited by 320
  8. 2016Molecular Graph Convolutions: Moving Beyond Fingerprintscited by 238
  9. 2018DeepDTA: Deep Drug-Target Binding Affinity Predictioncited by 173
  10. 2022SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentationcited by 171
  11. 2021Spherical Message Passing for 3D Graph Networkscited by 168
  12. 2021Geometric Deep Learning on Molecular Representationscited by 113
  13. 2022Transformer for Graphs: An Overview from Architecture Perspectivecited by 107
  14. 2019SMILES Transformer: Pre-trained Molecular Fingerprint for Low Data Drug Discoverycited by 90
  15. 2021SE(3) Equivariant Graph Neural Networks with Complete Local Framescited by 76
  16. 2019Molecular Property Prediction: A Multilevel Quantum Interactions Modeling Perspectivecited by 73
  17. 2021ProGCL: Rethinking Hard Negative Mining in Graph Contrastive Learningcited by 72
  18. 2021Co-learning: Learning from Noisy Labels with Self-supervisioncited by 51
  19. 2023SELFormer: Molecular Representation Learning via SELFIES Language Modelscited by 40
  20. 2017Deep Learning for Computational Chemistrycited by 38
  21. 2023Dink-Net: Neural Clustering on Large Graphscited by 38
  22. 2020Exploiting Neuron and Synapse Filter Dynamics in Spatial Temporal Learning of Deep Spiking Neural Networkcited by 37
  23. 2022A Systematic Survey of Chemical Pre-trained Modelscited by 34
  24. 2022A Survey of Pretraining on Graphs: Taxonomy, Methods, and Applicationscited by 23
  25. 2022Protein Language Models and Structure Prediction: Connection and Progressioncited by 17
  26. 2022Generative De Novo Protein Design with Global Contextcited by 16
  27. 2022CoSP: Co-supervised pretraining of pocket and ligandcited by 12
  28. 2022Using Context-to-Vector with Graph Retrofitting to Improve Word Embeddingscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2025Deep Learning Foundation Models from Classical Molecular Descriptorscited by 14
  2. 2024MaskMol: Knowledge-guided Molecular Image Pre-Training Framework for Activity Cliffscited by 2
  3. 2026Do Larger Models Really Win in Drug Discovery? A Benchmark Assessment of Model Scaling in AI-Driven Molecular Property and Activity Predictioncited by 2