# ControllableGPT: A Ground-Up Designed Controllable GPT for Molecule Optimization

arXiv: https://arxiv.org/abs/2502.10631 · HTML: https://mlsanity.com/paper/2502.10631/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Feb 2025.

## Built on by (most cited first, 4 of 4)

- [A Survey of Large Language Models for Text-Guided Molecular Discovery: from Molecule Generation to Optimization](https://mlsanity.com/paper/2505.16094/) (2025)
- [Monte Carlo Tree Diffusion with Multiple Experts for Protein Design](https://mlsanity.com/paper/2509.15796/) (2025)
- [FragmentGPT: A Unified GPT Model for Fragment Growing, Linking, and Merging in Molecular Design](https://mlsanity.com/paper/2509.11044/) (2025)
- [Leveraging generative hallucination and biophysics-informed modeling for unified biomolecular sequence-structure co-design](https://mlsanity.com/paper/2608.17381/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) employ three popular training approaches: Masked Language Models (MLM), Causal Language Models (CLM), and Sequence-to-Sequence Models (seq2seq). However, each approach has its strengths and limitations, and faces challenges in addressing specific tasks that require controllable and bidirectional generation, such as drug optimization. To address this challenge, inspired by the biological processes of growth and evolution, which involve the expansion, shrinking, and mutation of sequences, we introduce ControllableGPT. This initiative represents the first effort to combine the advantages of MLM, CLM, and seq2seq into a single unified, controllable GPT framework. It enables the precise management of specific locations and ranges within a sequence, allowing for expansion, reduction, or mutation over chosen or random lengths, while maintaining the integrity of any specified positions or subsequences. In this work, we designed ControllableGPT for drug optimization from the ground up, which included proposing the Causally Masked Seq2seq (CMS) objective, developing the training corpus, introducing a novel pre-training approach, and devising a unique generation process. We demonstrate the effectiveness and controllability of ControllableGPT by conducting experiments on drug optimization tasks for both viral and cancer benchmarks, surpassing competing baselines.