# Parallel-mentoring for Offline Model-based Optimization

arXiv: https://arxiv.org/abs/2309.11592 · HTML: https://mlsanity.com/paper/2309.11592/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Sep 2023.

## Built on by (most cited first, 26 of 26)

- [ParetoFlow: Guided Flows in Multi-Objective Optimization](https://mlsanity.com/paper/2412.03718/) (2024)
- [Guided Trajectory Generation with Diffusion Models for Offline Model-based Optimization](https://mlsanity.com/paper/2407.01624/) (2024)
- [Offline Model-Based Optimization: Comprehensive Review](https://mlsanity.com/paper/2503.17286/) (2025)
- [Offline Multi-Objective Optimization](https://mlsanity.com/paper/2406.03722/) (2024)
- [Improved Off-policy Reinforcement Learning in Biological Sequence Design](https://mlsanity.com/paper/2410.04461/) (2024)
- [Offline Model-Based Optimization by Learning to Rank](https://mlsanity.com/paper/2410.11502/) (2024)
- [Preference-Guided Diffusion for Multi-Objective Offline Optimization](https://mlsanity.com/paper/2503.17299/) (2025)
- [Design Editing for Offline Model-based Optimization](https://mlsanity.com/paper/2405.13964/) (2024)
- ["Noisier" Noise Contrastive Eestimation is (Almost) Maximum Likelihood](https://mlsanity.com/paper/2405.16730/) (2024)
- [Robust Guided Diffusion for Offline Black-Box Optimization](https://mlsanity.com/paper/2410.00983/) (2024)
- [Towards Universal Offline Black-Box Optimization via Learning Language Model Embeddings](https://mlsanity.com/paper/2506.07109/) (2025)
- [Boosting Offline Optimizers with Surrogate Sensitivity](https://mlsanity.com/paper/2503.04181/) (2025)
- [ROOT: Rethinking Offline Optimization as Distributional Translation via Probabilistic Bridge](https://mlsanity.com/paper/2509.16300/) (2025)
- [FoMEMO: Towards Foundation Models for Expensive Multi-objective Optimization](https://mlsanity.com/paper/2509.03244/) (2025)
- [SPREAD: Sampling-based Pareto front Refinement via Efficient Adaptive Diffusion](https://mlsanity.com/paper/2509.21058/) (2025)
- [Diffusion Large Language Models for Black-Box Optimization](https://mlsanity.com/paper/2601.14446/) (2026)
- [Pareto-Conditioned Diffusion Models for Offline Multi-Objective Optimization](https://mlsanity.com/paper/2602.00737/) (2026)
- [On the Learnability of Offline Model-Based Optimization: A Ranking Perspective](https://mlsanity.com/paper/2603.04000/) (2026)
- [From Function to Distribution Modeling: A PAC-Generative Approach to Offline Optimization](https://mlsanity.com/paper/2401.02019/) (2024)
- [AffinityFlow: Guided Flows for Antibody Affinity Maturation](https://mlsanity.com/paper/2502.10365/) (2025)
- [Learning Design-Score Manifold to Guide Diffusion Models for Offline Optimization](https://mlsanity.com/paper/2506.05680/) (2025)
- [Support-Proximity Augmented Diffusion Estimation for Offline Black-Box Optimization](https://mlsanity.com/paper/2605.11246/) (2026)
- [The Offline-Frontier Shift: Diagnosing Distributional Limits in Generative Multi-Objective Optimization](https://mlsanity.com/paper/2602.11126/) (2026)
- [Black-Box Optimization From Small Offline Datasets via Meta Learning with Synthetic Tasks](https://mlsanity.com/paper/2604.12325/) (2026)
- [Rethinking Learnability in Offline Data-driven Optimization](https://mlsanity.com/paper/2609.01493/) (2026)
- [ParetoTransport: Generative Optimization by Mass Transport Toward The Pareto Front](https://mlsanity.com/paper/2609.07706/) (2026)

## Abstract (arXiv)

We study offline model-based optimization to maximize a black-box objective function with a static dataset of designs and scores. These designs encompass a variety of domains, including materials, robots and DNA sequences. A common approach trains a proxy on the static dataset to approximate the black-box objective function and performs gradient ascent to obtain new designs. However, this often results in poor designs due to the proxy inaccuracies for out-of-distribution designs. Recent studies indicate that: (a) gradient ascent with a mean ensemble of proxies generally outperforms simple gradient ascent, and (b) a trained proxy provides weak ranking supervision signals for design selection. Motivated by (a) and (b), we propose textitparallel-mentoring as an effective and novel method that facilitates mentoring among parallel proxies, creating a more robust ensemble to mitigate the out-of-distribution issue. We focus on the three-proxy case and our method consists of two modules. The first module, textitvoting-based pairwise supervision, operates on three parallel proxies and captures their ranking supervision signals as pairwise comparison labels. These labels are combined through majority voting to generate consensus labels, which incorporate ranking supervision signals from all proxies and enable mutual mentoring. However, label noise arises due to possible incorrect consensus. To alleviate this, we introduce an textitadaptive soft-labeling module with soft-labels initialized as consensus labels. Based on bi-level optimization, this module fine-tunes proxies in the inner level and learns more accurate labels in the outer level to adaptively mentor proxies, resulting in a more robust ensemble. Experiments validate the effectiveness of our method. Our code is available here.