# SURF: Steering the Scalarization Weight to Uniformly Traverse the Pareto Front

arXiv: https://arxiv.org/abs/2605.20619 · HTML: https://mlsanity.com/paper/2605.20619/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 115 references that are in the map.
- First posted on arXiv 20 May 2026.

## Builds on (most cited first, 16 of 16)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Safe RLHF: Safe Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2310.12773/) (2023)
- [Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustment](https://mlsanity.com/paper/2402.10207/) (2024)
- [Smooth Tchebycheff Scalarization for Multi-Objective Optimization](https://mlsanity.com/paper/2402.19078/) (2024)
- [PARM: Multi-Objective Test-Time Alignment via Preference-Aware Autoregressive Reward Model](https://mlsanity.com/paper/2505.06274/) (2025)
- [Pareto Conditioned Networks](https://mlsanity.com/paper/2204.05036/) (2022)
- [Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyond](https://mlsanity.com/paper/2501.10945/) (2025)
- [Traversing Pareto Optimal Policies: Provably Efficient Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2407.17466/) (2024)
- [Pareto Merging: Multi-Objective Optimization for Preference-Aware Model Merging](https://mlsanity.com/paper/2408.12105/) (2024)
- [Multi-objective Large Language Model Alignment with Hierarchical Experts](https://mlsanity.com/paper/2505.20925/) (2025)
- [Projection Optimization: A General Framework for Multi-Objective and Multi-Group RLHF](https://mlsanity.com/paper/2502.15145/) (2025)
- [Policy-regularized Offline Multi-objective Reinforcement Learning](https://mlsanity.com/paper/2401.02244/) (2024)
- [Multi-objective Reinforcement learning from AI Feedback](https://mlsanity.com/paper/2406.07295/) (2024)
- [Optimization on Pareto sets: On a theory of multi-objective optimization](https://mlsanity.com/paper/2308.02145/) (2023)
- [Simultaneous Multi-objective Alignment Across Verifiable and Non-verifiable Rewards](https://mlsanity.com/paper/2510.01167/) (2025)

## Abstract (arXiv)

Scalarization is widely used in multi-objective optimization owing to its simplicity and scalability. In many applications, the goal is to generate solutions that represent diverse user preferences, ideally with uniform coverage of the Pareto front (PF). However, uniformly sampling scalarization weights usually induces non-uniform coverage of the PF. We explain this mismatch through the geometry of the scalarization path. As the weight varies, the corresponding solutions traverse the PF at a generally nonuniform speed. Integrating this speed gives cumulative arc length. After normalization by the total length, it defines a cumulative distribution function (CDF) on weights whose inverse selects points at equal arc-length intervals. Building on this insight, we propose SURF (Sampling Uniformly along the Pareto Front). For structured problems, including bi-objective bandits, we derive analytical expressions for this CDF map and the resulting PF-aware weight sampling rule. For general problems, SURF alternates between CDF reconstruction and weight sampling. Theoretically, we show that under provable conditions for bi-objective LS, SURF's CDF reconstruction error converges linearly to an O(N-2) discretization floor. Empirically, experiments on bandits, multi-objective-gymnasium, and multi-objective LLM alignment demonstrate that SURF efficiently achieves more uniform PF coverage than baselines.