Inverse Design for Conditional Distribution Matching

Generative models are powerful tools for sampling from a learned distribution P(Y mid X), and inverse-design methods invert this map to find an input x that produces a desired point output y^*. However, many design goals are naturally distributional rather than pointwise, incorporating the inherent uncertainty of Y and targeting a specific form for it, a task not addressed by standard inverse design. To address this issue we introduce Conditional Distribution Matching (CDM), a new inverse-design problem class in generative modeling: given a joint distribution P(X, Y) and a target distribution G(Y), find an input x^* whose induced conditional distribution P(Y mid X = x^*) matches G. We formally define two variants: Conditional Distribution Matching Sampling (CDMS) and Conditional Distribution Matching Optimization (CDMO). To solve these problems, we propose MLGD-F (Matching-Loss Guided Diffusion with a Fast inner sampler), a plug-and-play inference-time algorithm that combines a pretrained score-based diffusion model with a pretrained fast conditional sampler, requiring no additional training or fine-tuning. By leveraging single-step conditional sampling, MLGD-F enables tractable gradient computation, making the estimation of P(Y mid X) both memory-efficient and computationally lightweight. We validate MLGD-F on synthetic benchmarks, structured image transformations, and generative editing optimization, demonstrating reliable recovery of inputs whose conditional distributions match diverse user-specified targets, including discrete mixtures and continuous low-rank supports.