mlsanityData to 14 Oct 2026

Ensemble Model With Bert,Roberta and Xlnet For Molecular property prediction

arXiv 2406.06553 ↗First posted 30 May 2024cs.LG cs.AI physics.chem-phcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper presents a novel approach for predicting molecular properties with high accuracy without the need for extensive pre-training. Employing ensemble learning and supervised fine-tuning of BERT, RoBERTa, and XLNet, our method demonstrates significant effectiveness compared to existing advanced models. Crucially, it addresses the issue of limited computational resources faced by experimental groups, enabling them to accurately predict molecular properties. This innovation provides a cost-effective and resource-efficient solution, potentially advancing further research in the molecular domain.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 23 of its 67 references that are in the map. First posted on arXiv 30 May 2024.

Builds on

23 of 67 references.

References · 23

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  5. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  6. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,051
  7. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 765
  8. 2015Convolutional Networks on Graphs for Learning Molecular Fingerprintscited by 754
  9. 2019Analyzing Learned Molecular Representations for Property Predictioncited by 375
  10. 2020ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Predictioncited by 337
  11. 2021Large-Scale Chemical Language Representations Capture Molecular Structure and Propertiescited by 206
  12. 2021Motif-based Graph Self-Supervised Learning for Molecular Property Predictioncited by 159
11 more
  1. 2020Pre-training is a Hot Topic: Contextualized Document Embeddings Improve Topic Coherencecited by 121
  2. 2021Exploring the Limits of Large Scale Pre-trainingcited by 104
  3. 2020Molecular representation learning with language models and domain-relevant auxiliary taskscited by 94
  4. 2019SMILES Transformer: Pre-trained Molecular Fingerprint for Low Data Drug Discoverycited by 90
  5. 2020How Useful is Self-Supervised Pretraining for Visual Tasks?cited by 80
  6. 2022Regression Transformer: Concurrent sequence regression and generation for molecular language modelingcited by 50
  7. 2018Conditional molecular design with deep generative modelscited by 44
  8. 2020What the [MASK]? Making Sense of Language-Specific BERT Modelscited by 44
  9. 2017Deep Learning for Computational Chemistrycited by 38
  10. 2017SMILES2Vec: An Interpretable General-Purpose Deep Neural Network for Predicting Chemical Propertiescited by 30
  11. 2019All SMILES Variational Autoencodercited by 15

Built on by

Not cited yet.