# Improving Small Molecule Generation using Mutual Information Machine

arXiv: https://arxiv.org/abs/2208.09016 · HTML: https://mlsanity.com/paper/2208.09016/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 63 references that are in the map.
- First posted on arXiv 18 Aug 2022.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Perceiver: General Perception with Iterative Attention](https://mlsanity.com/paper/2103.03206/) (2021)
- [Automatic chemical design using a data-driven continuous representation of molecules](https://mlsanity.com/paper/1610.02415/) (2016)
- [Junction Tree Variational Autoencoder for Molecular Graph Generation](https://mlsanity.com/paper/1802.04364/) (2018)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation](https://mlsanity.com/paper/1806.02473/) (2018)
- [GraphAF: a Flow-based Autoregressive Model for Molecular Graph Generation](https://mlsanity.com/paper/2001.09382/) (2020)
- [GuacaMol: Benchmarking Models for De Novo Molecular Design](https://mlsanity.com/paper/1811.09621/) (2018)
- [Molecular De Novo Design through Deep Reinforcement Learning](https://mlsanity.com/paper/1704.07555/) (2017)
- [Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development](https://mlsanity.com/paper/2102.09548/) (2021)
- [NeMo: a toolkit for building AI applications using Neural Modules](https://mlsanity.com/paper/1909.09577/) (2019)
- [Large-Scale Chemical Language Representations Capture Molecular Structure and Properties](https://mlsanity.com/paper/2106.09553/) (2021)
- [Optimization of Molecules via Deep Reinforcement Learning](https://mlsanity.com/paper/1810.08678/) (2018)
- [MoFlow: An Invertible Flow Model for Generating Molecular Graphs](https://mlsanity.com/paper/2006.10137/) (2020)
- [GraphDF: A Discrete Flow Model for Molecular Graph Generation](https://mlsanity.com/paper/2102.01189/) (2021)
- [MARS: Markov Molecular Sampling for Multi-objective Drug Discovery](https://mlsanity.com/paper/2103.10432/) (2021)
- [Learning Multimodal Graph-to-Graph Translation for Molecular Optimization](https://mlsanity.com/paper/1812.01070/) (2018)
- [Preventing Posterior Collapse with delta-VAEs](https://mlsanity.com/paper/1901.03416/) (2019)
- [Augmenting Genetic Algorithms with Deep Neural Networks for Exploring the Chemical Space](https://mlsanity.com/paper/1909.11655/) (2019)
- [Optimizing Molecules using Efficient Queries from Property Evaluations](https://mlsanity.com/paper/2011.01921/) (2020)
- [JANUS: Parallel Tempered Genetic Algorithm Guided by Deep Neural Networks for Inverse Molecular Design](https://mlsanity.com/paper/2106.04011/) (2021)
- [Autonomous discovery in the chemical sciences part II: Outlook](https://mlsanity.com/paper/2003.13755/) (2020)
- [MolGrow: A Graph Normalizing Flow for Hierarchical Molecular Generation](https://mlsanity.com/paper/2106.05856/) (2021)
- [A deep-learning view of chemical space designed to facilitate drug discovery](https://mlsanity.com/paper/2002.02948/) (2020)
- [Learning Space Partitions for Path Planning](https://mlsanity.com/paper/2106.10544/) (2021)
- [Fragment-based Sequential Translation for Molecular Optimization](https://mlsanity.com/paper/2111.01009/) (2021)
- [SentenceMIM: A Latent Variable Language Model](https://mlsanity.com/paper/2003.02645/) (2020)

## Built on by (most cited first, 10 of 10)

- [nach0: Multimodal Natural and Chemical Languages Foundation Model](https://mlsanity.com/paper/2311.12410/) (2023)
- [CoarsenConf: Equivariant Coarsening with Aggregated Attention for Molecular Conformer Generation](https://mlsanity.com/paper/2306.14852/) (2023)
- [TRIDENT: Tri-Modal Molecular Representation Learning with Taxonomic Annotations and Local Correspondence](https://mlsanity.com/paper/2506.21028/) (2025)
- [Diffusion Models with Double Guidance: Generate with aggregated datasets](https://mlsanity.com/paper/2505.13213/) (2025)
- [Technical Implementation of Tippy: Multi-Agent Architecture and System Design for Drug Discovery Laboratory Automation](https://mlsanity.com/paper/2507.17852/) (2025)
- [Probabilistic Contrastive Pretraining for Multi-task ADME Property Prediction](https://mlsanity.com/paper/2606.11508/) (2026)
- [ChatMol: A Versatile Molecule Designer Based on the Numerically Enhanced Large Language Model](https://mlsanity.com/paper/2502.19794/) (2025)
- [Pairwise Optimal Transports for Training All-to-All Flow-Based Condition Transfer Model](https://mlsanity.com/paper/2504.03188/) (2025)
- [Contrastive Mutual Information Learning: Toward Robust Representations without Positive-Pair Augmentations](https://mlsanity.com/paper/2509.21511/) (2025)
- [Contrastive MIM: A Contrastive Mutual Information Framework for Unified Generative and Discriminative Representation Learning](https://mlsanity.com/paper/2502.19642/) (2025)

## Abstract (arXiv)

We address the task of controlled generation of small molecules, which entails finding novel molecules with desired properties under certain constraints (e.g., similarity to a reference molecule). Here we introduce MolMIM, a probabilistic auto-encoder for small molecule drug discovery that learns an informative and clustered latent space. MolMIM is trained with Mutual Information Machine (MIM) learning, and provides a fixed length representation of variable length SMILES strings. Since encoder-decoder models can learn representations with ``holes'' of invalid samples, here we propose a novel extension to the training procedure which promotes a dense latent space, and allows the model to sample valid molecules from random perturbations of latent codes. We provide a thorough comparison of MolMIM to several variable-size and fixed-size encoder-decoder models, demonstrating MolMIM's superior generation as measured in terms of validity, uniqueness, and novelty. We then utilize CMA-ES, a naive black-box and gradient free search algorithm, over MolMIM's latent space for the task of property guided molecule optimization. We achieve state-of-the-art results in several constrained single property optimization tasks as well as in the challenging task of multi-objective optimization, improving over previous success rate SOTA by more than 5% . We attribute the strong results to MolMIM's latent representation which clusters similar molecules in the latent space, whereas CMA-ES is often used as a baseline optimization method. We also demonstrate MolMIM to be favourable in a compute limited regime, making it an attractive model for such cases.