mlsanityData to 14 Oct 2026

Synergistic Benefits of Joint Molecule Generation and Property Prediction

arXiv 2504.16559 ↗First posted 23 Apr 2025cs.LG q-bio.QMcited by 5

Not on the map. Why?

Abstract (arXiv)

Modeling the joint distribution of data samples and their properties allows to construct a single model for both data generation and property prediction, with synergistic benefits reaching beyond purely generative or predictive models. However, training joint models presents daunting architectural and optimization challenges. Here, we propose Hyformer, a transformer-based joint model that successfully blends the generative and predictive functionalities, using an alternating attention mechanism and a joint pre-training scheme. We show that Hyformer is simultaneously optimized for molecule generation and property prediction, while exhibiting synergistic benefits in conditional sampling, out-of-distribution property prediction and representation learning. Finally, we demonstrate the benefits of joint learning in a drug design use case of discovering novel antimicrobial~peptides.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 36 of its 88 references that are in the map. First posted on arXiv 23 Apr 2025.

Builds on

36 of 88 references.

References · 36

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  6. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,095
  7. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,051
  8. 2019Strategies for Pre-training Graph Neural Networkscited by 971
  9. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 765
  10. 2022Equivariant Diffusion for Molecule Generation in 3Dcited by 677
  11. 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 670
  12. 2018Junction Tree Variational Autoencoder for Molecular Graph Generationcited by 652
24 more
  1. 2019Your Classifier is Secretly an Energy Based Model and You Should Treat it Like Onecited by 405
  2. 2019Analyzing Learned Molecular Representations for Property Predictioncited by 375
  3. 2018GuacaMol: Benchmarking Models for De Novo Molecular Designcited by 297
  4. 2021Pre-training Molecular Graph Representation with 3D Geometrycited by 248
  5. 2021ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Predictioncited by 204
  6. 2018Constrained Graph Variational Autoencoders for Molecule Designcited by 185
  7. 2021GraphDF: A Discrete Flow Model for Molecular Graph Generationcited by 180
  8. 20233D Equivariant Diffusion for Target-Aware Molecule Generation and Affinity Predictioncited by 177
  9. 2023Controlled Decoding from Language Modelscited by 135
  10. 2020Molecular representation learning with language models and domain-relevant auxiliary taskscited by 94
  11. 2018N-Gram Graph: Simple Unsupervised Representation for Graphs, with Applications to Moleculescited by 93
  12. 2021GraphEBM: Molecular Graph Generation with Energy-Based Modelscited by 85
  13. 2020Mapping the Space of Chemical Reactions Using Attention-Based Neural Networkscited by 70
  14. 2021Keeping it Simple: Language Models can learn Complex Molecular Distributionscited by 68
  15. 2022Benchmarking Graphormer on Large-Scale Molecular Modeling Datasetscited by 64
  16. 2017SMILES Enumeration as Data Augmentation for Neural Network Modeling of Moleculescited by 59
  17. 2022Regression Transformer: Concurrent sequence regression and generation for molecular language modelingcited by 50
  18. 2023De Novo Molecular Generation via Connection-aware Motif Miningcited by 46
  19. 2019Hybrid Models with Deep and Invertible Featurescited by 41
  20. 2024UniGEM: A Unified Approach to Generation and Property Prediction for Moleculescited by 21
  21. 2023Lo-Hi: Practical ML Drug Discovery Benchmarkcited by 10
  22. 2024A Graph is Worth K Words: Euclideanizing Graph using Pure Transformercited by 10
  23. 2024Rethinking Specificity in SBDD: Leveraging Delta Score and Energy-Guided Diffusioncited by 9
  24. 2024Transformers for molecular property prediction: Lessons learned from the past five yearscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 1.

5 papers.

  1. 2025Enhancing next token prediction based pre-training for jet foundation modelscited by 3
  2. 2025Valid Property-Enhanced Contrastive Learning for Targeted Optimization & Resampling for Novel Drug Designcited by 0
  3. 2025seqme: a Python library for evaluating biological sequence designcited by 0
  4. 2025Freeze, Diffuse, Decode: Task-Aware Adaptation of Transformer Embeddings for Antimicrobial Peptide Designcited by 0
  5. 2026Sample Efficient Generative Molecular Optimization with Joint Self-Improvementcited by 0