# Learning to design without prior data: Discovering generalizable design strategies using deep learning and tree search

arXiv: https://arxiv.org/abs/2211.15068 · HTML: https://mlsanity.com/paper/2211.15068/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 71 references that are in the map.
- First posted on arXiv 28 Nov 2022.

## Builds on (most cited first, 25 of 25)

- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning](https://mlsanity.com/paper/1910.10897/) (2019)
- [Deep Reinforcement Learning at the Edge of the Statistical Precipice](https://mlsanity.com/paper/2108.13264/) (2021)
- [An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution](https://mlsanity.com/paper/1807.03247/) (2018)
- [Thinking Fast and Slow with Deep Learning and Tree Search](https://mlsanity.com/paper/1705.08439/) (2017)
- [Combinatorial Optimization with Graph Convolutional Networks and Guided Tree Search](https://mlsanity.com/paper/1810.10659/) (2018)
- [A Closer Look at Invalid Action Masking in Policy Gradient Algorithms](https://mlsanity.com/paper/2006.14171/) (2020)
- [Monte Carlo Tree Search: A Review of Recent Modifications and Applications](https://mlsanity.com/paper/2103.04931/) (2021)
- [Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networks](https://mlsanity.com/paper/2106.04537/) (2021)
- [Deep Generative Models in Engineering Design: A Review](https://mlsanity.com/paper/2110.10863/) (2021)
- [Learning and Planning in Complex Action Spaces](https://mlsanity.com/paper/2104.06303/) (2021)
- [Deep Reinforcement Learning in Parameterized Action Space](https://mlsanity.com/paper/1511.04143/) (2015)
- [Monte-Carlo Tree Search as Regularized Policy Optimization](https://mlsanity.com/paper/2007.12509/) (2020)
- [Learning to Search Better Than Your Teacher](https://mlsanity.com/paper/1502.02206/) (2015)
- [Learning what you can do before doing anything](https://mlsanity.com/paper/1806.09655/) (2018)
- [MO-PaDGAN: Reparameterizing Engineering Designs for Augmented Multi-objective Optimization](https://mlsanity.com/paper/2009.07110/) (2020)
- [A Deep Reinforcement Learning Approach for Global Routing](https://mlsanity.com/paper/1906.08809/) (2019)
- [A0C: Alpha Zero in Continuous Action Space](https://mlsanity.com/paper/1805.09613/) (2018)
- [3D Shape Synthesis for Conceptual Design and Optimization Using Variational Autoencoders](https://mlsanity.com/paper/1904.07964/) (2019)
- [Watch the Unobserved: A Simple Approach to Parallelizing Monte Carlo Tree Search](https://mlsanity.com/paper/1810.11755/) (2018)
- [Attention Routing: track-assignment detailed routing using attention-based reinforcement learning](https://mlsanity.com/paper/2004.09473/) (2020)
- [Learning to design from humans: Imitating human designers through deep learning](https://mlsanity.com/paper/1907.11813/) (2019)
- [Circuit Routing Using Monte Carlo Tree Search and Deep Neural Networks](https://mlsanity.com/paper/2006.13607/) (2020)
- [Design Strategy Network: A deep hierarchical framework to represent generative design strategies in complex action spaces](https://mlsanity.com/paper/2110.03760/) (2021)
- [Goal-Directed Design Agents: Integrating Visual Imitation with One-Step Lookahead Optimization for Generative Design](https://mlsanity.com/paper/2110.03223/) (2021)

## Built on by (most cited first, 5 of 5)

- [Data-Driven Design for Metamaterials and Multiscale Systems: A Review](https://mlsanity.com/paper/2307.05506/) (2023)
- [Beyond Games: A Systematic Review of Neural Monte Carlo Tree Search Applications](https://mlsanity.com/paper/2303.08060/) (2023)
- [TransAxx: Efficient Transformers with Approximate Computing](https://mlsanity.com/paper/2402.07545/) (2024)
- [Bridging Design Gaps: A Parametric Data Completion Approach With Graph Guided Diffusion Models](https://mlsanity.com/paper/2406.11934/) (2024)
- [Adaptive Multi-Fidelity Reinforcement Learning for Variance Reduction in Engineering Design Optimization](https://mlsanity.com/paper/2503.18229/) (2025)

## Abstract (arXiv)

Building an AI agent that can design on its own has been a goal since the 1980s. Recently, deep learning has shown the ability to learn from large-scale data, enabling significant advances in data-driven design. However, learning over prior data limits us only to solve problems that have been solved before and biases data-driven learning towards existing solutions. The ultimate goal for a design agent is the ability to learn generalizable design behavior in a problem space without having seen it before. We introduce a self-learning agent framework in this work that achieves this goal. This framework integrates a deep policy network with a novel tree search algorithm, where the tree search explores the problem space, and the deep policy network leverages self-generated experience to guide the search further. This framework first demonstrates an ability to discover high-performing generative strategies without any prior data, and second, it illustrates a zero-shot generalization of generative strategies across various unseen boundary conditions. This work evaluates the effectiveness and versatility of the framework by solving multiple versions of two engineering design problems without retraining. Overall, this paper presents a methodology to self-learn high-performing and generalizable problem-solving behavior in an arbitrary problem space, circumventing the needs for expert data, existing solutions, and problem-specific learning.