# Automated patent extraction powers generative modeling in focused chemical spaces

arXiv: https://arxiv.org/abs/2303.08272 · HTML: https://mlsanity.com/paper/2303.08272/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 64 references that are in the map.
- First posted on arXiv 14 Mar 2023.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [MoleculeNet: A Benchmark for Molecular Machine Learning](https://mlsanity.com/paper/1703.00564/) (2017)
- [Automatic chemical design using a data-driven continuous representation of molecules](https://mlsanity.com/paper/1610.02415/) (2016)
- [Junction Tree Variational Autoencoder for Molecular Graph Generation](https://mlsanity.com/paper/1802.04364/) (2018)
- [Analyzing Learned Molecular Representations for Property Prediction](https://mlsanity.com/paper/1904.01561/) (2019)
- [ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction](https://mlsanity.com/paper/2010.09885/) (2020)
- [GuacaMol: Benchmarking Models for De Novo Molecular Design](https://mlsanity.com/paper/1811.09621/) (2018)
- [Grammar Variational Autoencoder](https://mlsanity.com/paper/1703.01925/) (2017)
- [Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Models](https://mlsanity.com/paper/1811.12823/) (2018)
- [Molecular De Novo Design through Deep Reinforcement Learning](https://mlsanity.com/paper/1704.07555/) (2017)
- [Fr'echet ChemNet Distance: A metric for generative models for molecules in drug discovery](https://mlsanity.com/paper/1803.09518/) (2018)
- [Sample Efficiency Matters: A Benchmark for Practical Molecular Optimization](https://mlsanity.com/paper/2206.12411/) (2022)
- [Interactive Supercomputing on 40,000 Cores for Machine Learning and Data Analysis](https://mlsanity.com/paper/1807.07814/) (2018)
- [Molecule Attention Transformer](https://mlsanity.com/paper/2002.08264/) (2020)
- [Keeping it Simple: Language Models can learn Complex Molecular Distributions](https://mlsanity.com/paper/2112.03041/) (2021)
- [Reinforcement Learning for Molecular Design Guided by Quantum Mechanics](https://mlsanity.com/paper/2002.07717/) (2020)
- [Improving black-box optimization in VAE latent space using decoder uncertainty](https://mlsanity.com/paper/2107.00096/) (2021)
- [LIMO: Latent Inceptionism for Targeted Molecule Generation](https://mlsanity.com/paper/2206.09010/) (2022)
- [Deep learning for molecular design - a review of the state of the art](https://mlsanity.com/paper/1903.04388/) (2019)
- [Molecular Generation with Recurrent Neural Networks (RNNs)](https://mlsanity.com/paper/1705.04612/) (2017)
- [Generative Models for Automatic Chemical Design](https://mlsanity.com/paper/1907.01632/) (2019)
- [Biases in In Silico Evaluation of Molecular Optimization Methods and Bias-Reduced Evaluation Methodology](https://mlsanity.com/paper/2201.12163/) (2022)
- [mkite: A distributed computing platform for high-throughput materials simulations](https://mlsanity.com/paper/2301.08841/) (2023)

## Built on by (most cited first, 4 of 4)

- [InstructPatentGPT: Training patent language models to follow instructions with human feedback](https://mlsanity.com/paper/2406.16897/) (2024)
- [Mining Patents with Large Language Models Elucidates the Chemical Function Landscape](https://mlsanity.com/paper/2309.08765/) (2023)
- [Generative Marginalization Models](https://mlsanity.com/paper/2310.12920/) (2023)
- [Efficient Generation of Molecular Clusters with Dual-Scale Equivariant Flow Matching](https://mlsanity.com/paper/2410.07539/) (2024)

## Abstract (arXiv)

Deep generative models have emerged as an exciting avenue for inverse molecular design, with progress coming from the interplay between training algorithms and molecular representations. One of the key challenges in their applicability to materials science and chemistry has been the lack of access to sizeable training datasets with property labels. Published patents contain the first disclosure of new materials prior to their publication in journals, and are a vast source of scientific knowledge that has remained relatively untapped in the field of data-driven molecular design. Because patents are filed seeking to protect specific uses, molecules in patents can be considered to be weakly labeled into application classes. Furthermore, patents published by the US Patent and Trademark Office (USPTO) are downloadable and have machine-readable text and molecular structures. In this work, we train domain-specific generative models using patent data sources by developing an automated pipeline to go from USPTO patent digital files to the generation of novel candidates with minimal human intervention. We test the approach on two in-class extracted datasets, one in organic electronics and another in tyrosine kinase inhibitors. We then evaluate the ability of generative models trained on these in-class datasets on two categories of tasks (distribution learning and property optimization), identify strengths and limitations, and suggest possible explanations and remedies that could be used to overcome these in practice.