# Learning from flowsheets: A generative transformer model for autocompletion of flowsheets

arXiv: https://arxiv.org/abs/2208.00859 · HTML: https://mlsanity.com/paper/2208.00859/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 53 references that are in the map.
- First posted on arXiv 1 Aug 2022.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction](https://mlsanity.com/paper/2010.09885/) (2020)
- [Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models](https://mlsanity.com/paper/1610.02424/) (2016)
- [Molecular Transformer - A Model for Uncertainty-Calibrated Chemical Reaction Prediction](https://mlsanity.com/paper/1811.02633/) (2018)
- ["Found in Translation": Predicting Outcomes of Complex Organic Chemistry Reactions using Neural Sequence-to-Sequence Models](https://mlsanity.com/paper/1711.04810/) (2017)
- [Generating High-Quality and Informative Conversation Responses with Sequence-to-Sequence Models](https://mlsanity.com/paper/1701.03185/) (2017)
- [Global Deterministic Optimization with Artificial Neural Networks Embedded](https://mlsanity.com/paper/1801.07114/) (2018)
- [A Comparison of Transformer and Recurrent Neural Networks on Multilingual Neural Machine Translation](https://mlsanity.com/paper/1806.06957/) (2018)
- [Automated Synthesis of Steady-State Continuous Processes using Reinforcement Learning](https://mlsanity.com/paper/2101.04422/) (2021)
- [Flowsheet synthesis through hierarchical reinforcement learning and graph neural networks](https://mlsanity.com/paper/2207.12051/) (2022)
- [SFILES 2.0: An extended text-based flowsheet representation](https://mlsanity.com/paper/2208.00778/) (2022)
- [Deep Reinforcement Learning for Process Synthesis](https://mlsanity.com/paper/2009.13265/) (2020)

## Built on by (most cited first, 10 of 10)

- [Deep reinforcement learning for process design: Review and perspective](https://mlsanity.com/paper/2308.07822/) (2023)
- [Generative AI and Process Systems Engineering: The Next Frontier](https://mlsanity.com/paper/2402.10977/) (2024)
- [Toward autocorrection of chemical process flowsheets using large language models](https://mlsanity.com/paper/2312.02873/) (2023)
- [Introducing Hybrid Modeling with Time-series-Transformers: A Comparative Study of Series and Parallel Approach in Batch Crystallization](https://mlsanity.com/paper/2308.05749/) (2023)
- [Graph-to-SFILES: Control structure prediction from process topologies using generative artificial intelligence](https://mlsanity.com/paper/2412.00508/) (2024)
- [CrystalGPT: Enhancing system-to-system transferability in crystallization prediction and control using time-series-transformers](https://mlsanity.com/paper/2306.03099/) (2023)
- [Federated Learning from Molecules to Processes: A Perspective](https://mlsanity.com/paper/2506.18525/) (2025)
- [Data augmentation for machine learning of chemical process flowsheets](https://mlsanity.com/paper/2302.03379/) (2023)
- [LLMs in Process Diagram Engineering: From Optimal PFDs to Validated P&IDs](https://mlsanity.com/paper/2608.11220/) (2026)
- [Atoms to Processes: The Role of Artificial Intelligence and Machine Learning in Chemical Engineering](https://mlsanity.com/paper/2610.02014/) (2026)

## Abstract (arXiv)

We propose a novel method enabling autocompletion of chemical flowsheets. This idea is inspired by the autocompletion of text. We represent flowsheets as strings using the text-based SFILES 2.0 notation and learn the grammatical structure of the SFILES 2.0 language and common patterns in flowsheets using a transformer-based language model. We pre-train our model on synthetically generated flowsheet topologies to learn the flowsheet language grammar. Then, we fine-tune our model in a transfer learning step on real flowsheet topologies. Finally, we use the trained model for causal language modeling to autocomplete flowsheets. Eventually, the proposed method can provide chemical engineers with recommendations during interactive flowsheet synthesis. The results demonstrate a high potential of this approach for future AI-assisted process synthesis but also reveal the limitations at the present state and the next steps that need to be taken to deploy this technique in realistic flowsheet synthesis scenarios.