# e-SimFT: Alignment of Generative Models with Simulation Feedback for Pareto-Front Design Exploration

arXiv: https://arxiv.org/abs/2502.02628 · HTML: https://mlsanity.com/paper/2502.02628/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 11 of its 24 references that are in the map.
- First posted on arXiv 4 Feb 2025.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Diffusion and multimodal model alignment](https://mlsanity.com/alignment/diffusion-and-multimodal-model-alignment/)

## Builds on (most cited first, 11 of 11)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [KTO: Model Alignment as Prospect Theoretic Optimization](https://mlsanity.com/paper/2402.01306/) (2024)
- [Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustment](https://mlsanity.com/paper/2402.10207/) (2024)
- [RLTF: Reinforcement Learning from Unit Test Feedback](https://mlsanity.com/paper/2307.04349/) (2023)
- [Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment](https://mlsanity.com/paper/2402.19085/) (2024)
- [Panacea: Pareto Alignment via Preference Adaptation for LLMs](https://mlsanity.com/paper/2402.02030/) (2024)
- [Contrastive Preference Learning: Learning from Human Feedback without RL](https://mlsanity.com/paper/2310.13639/) (2023)
- [RLSF: Fine-tuning LLMs via Symbolic Feedback](https://mlsanity.com/paper/2405.16661/) (2024)
- [Multi-objective Reinforcement learning from AI Feedback](https://mlsanity.com/paper/2406.07295/) (2024)
- [Deep Generative Model for Mechanical System Configuration Design](https://mlsanity.com/paper/2409.06016/) (2024)

## Built on by (most cited first, 2 of 2)

- [Aligning Constraint Generation with Design Intent in Parametric CAD](https://mlsanity.com/paper/2504.13178/) (2025)
- [MechaFormer: Sequence Learning for Kinematic Mechanism Design Automation](https://mlsanity.com/paper/2508.09005/) (2025)

## Abstract (arXiv)

Deep generative models have recently shown success in solving complex engineering design problems where models predict solutions that address the design requirements specified as input. However, there remains a challenge in aligning such models for effective design exploration. For many design problems, finding a solution that meets all the requirements is infeasible. In such a case, engineers prefer to obtain a set of Pareto optimal solutions with respect to those requirements, but uniform sampling of generative models may not yield a useful Pareto front. To address this gap, we introduce a new framework for Pareto-front design exploration with simulation fine-tuned generative models. First, the framework adopts preference alignment methods developed for Large Language Models (LLMs) and showcases the first application in fine-tuning a generative model for engineering design. The important distinction here is that we use a simulator instead of humans to provide accurate and scalable feedback. Next, we propose epsilon-sampling, inspired by the epsilon-constraint method used for Pareto-front generation with classical optimization algorithms, to construct a high-quality Pareto front with the fine-tuned models. Our framework, named e-SimFT, is shown to produce better-quality Pareto fronts than existing multi-objective alignment methods.