# Automated Synthesis of Steady-State Continuous Processes using Reinforcement Learning

arXiv: https://arxiv.org/abs/2101.04422 · HTML: https://mlsanity.com/paper/2101.04422/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 36 references that are in the map.
- First posted on arXiv 12 Jan 2021.

## Builds on (most cited first, 3 of 3)

- [Deep Reinforcement Learning in Parameterized Action Space](https://mlsanity.com/paper/1511.04143/) (2015)
- [Parametrized Deep Q-Networks Learning: Reinforcement Learning with Discrete-Continuous Hybrid Action Space](https://mlsanity.com/paper/1810.06394/) (2018)
- [Continuous-Discrete Reinforcement Learning for Hybrid Control in Robotics](https://mlsanity.com/paper/2001.00449/) (2020)

## Built on by (most cited first, 10 of 10)

- [Flowsheet synthesis through hierarchical reinforcement learning and graph neural networks](https://mlsanity.com/paper/2207.12051/) (2022)
- [Learning from flowsheets: A generative transformer model for autocompletion of flowsheets](https://mlsanity.com/paper/2208.00859/) (2022)
- [Deep reinforcement learning for process design: Review and perspective](https://mlsanity.com/paper/2308.07822/) (2023)
- [Transfer learning for process design with reinforcement learning](https://mlsanity.com/paper/2302.03375/) (2023)
- [Policy-Based Self-Competition for Planning Problems](https://mlsanity.com/paper/2306.04403/) (2023)
- [Federated Learning from Molecules to Processes: A Perspective](https://mlsanity.com/paper/2506.18525/) (2025)
- [Sketch2Simulation: Automating Flowsheet Generation via Multi Agent Large Language Models](https://mlsanity.com/paper/2603.24629/) (2026)
- [Synthesis of separation processes with reinforcement learning](https://mlsanity.com/paper/2211.04327/) (2022)
- [Deep reinforcement learning uncovers processes for separating azeotropic mixtures without prior knowledge](https://mlsanity.com/paper/2310.06415/) (2023)
- [Enhanced Reinforcement Learning-based Process Synthesis via Quantum Computing](https://mlsanity.com/paper/2605.21213/) (2026)

## Abstract (arXiv)

Automated flowsheet synthesis is an important field in computer-aided process engineering. The present work demonstrates how reinforcement learning can be used for automated flowsheet synthesis without any heuristics of prior knowledge of conceptual design. The environment consists of a steady-state flowsheet simulator that contains all physical knowledge. An agent is trained to take discrete actions and sequentially built up flowsheets that solve a given process problem. A novel method named SynGameZero is developed to ensure good exploration schemes in the complex problem. Therein, flowsheet synthesis is modelled as a game of two competing players. The agent plays this game against itself during training and consists of an artificial neural network and a tree search for forward planning. The method is applied successfully to a reaction-distillation process in a quaternary system.