Few-Shot Design Optimization by Exploiting Auxiliary Information

Many real-world design problems involve optimizing an expensive black-box function f(x), such as hardware design or drug discovery. Bayesian Optimization has emerged as a sample-efficient framework for this problem. However, the basic setting considered by these methods is simplified compared to real-world experimental setups, where experiments often generate a wealth of useful information. We introduce a new setting where an experiment generates high-dimensional auxiliary information h(x) along with the performance measure f(x); moreover, a history of previously solved tasks from the same task family is available for accelerating optimization. A key challenge of our setting is learning how to represent and utilize h(x) for efficiently solving new optimization tasks beyond the task history. We develop a novel approach for this setting based on a neural model which predicts f(x) for unseen designs given a few-shot context containing observations of h(x). We evaluate our method on two challenging domains, robotic hardware design and neural network hyperparameter tuning, and introduce a novel design problem and large-scale benchmark for the former. On both domains, our method utilizes auxiliary feedback effectively to achieve more accurate few-shot prediction and faster optimization of design tasks, significantly outperforming several methods for multi-task optimization.