# Improving Pareto Front Learning via Multi-Sample Hypernetworks

arXiv: https://arxiv.org/abs/2212.01130 · HTML: https://mlsanity.com/paper/2212.01130/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 41 references that are in the map.
- First posted on arXiv 2 Dec 2022.

## Builds on (most cited first, 18 of 18)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [Multi-Task Learning as Multi-Objective Optimization](https://mlsanity.com/paper/1810.04650/) (2018)
- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [Pareto Multi-Task Learning](https://mlsanity.com/paper/1912.12854/) (2019)
- [Learning the Pareto Front with Hypernetworks](https://mlsanity.com/paper/2010.04104/) (2020)
- [Efficient Continuous Pareto Exploration in Multi-Task Learning](https://mlsanity.com/paper/2006.16434/) (2020)
- [Controllable Pareto Multi-Task Learning](https://mlsanity.com/paper/2010.06313/) (2020)
- [Scalable Pareto Front Approximation for Deep Multi-Objective Learning](https://mlsanity.com/paper/2103.13392/) (2021)
- [Multi-Gradient Descent for Multi-Objective Recommender Systems](https://mlsanity.com/paper/2001.00846/) (2019)
- [Multi-Objective Learning to Predict Pareto Fronts Using Hypervolume Maximization](https://mlsanity.com/paper/2102.04523/) (2021)
- [Rank and Rate: Multi-task Learning for Recommender Systems](https://mlsanity.com/paper/1807.11698/) (2018)
- [A Monotonic Optimization Approach for Solving Strictly Quasiconvex Multiobjective Programming Problems](https://mlsanity.com/paper/2003.07549/) (2020)
- [Multi-objective Optimization by Uncrowded Hypervolume Gradient Ascent](https://mlsanity.com/paper/2007.04846/) (2020)

## Built on by (most cited first, 16 of 16)

- [A Brief Review of Hypernetworks in Deep Learning](https://mlsanity.com/paper/2306.06955/) (2023)
- [Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyond](https://mlsanity.com/paper/2501.10945/) (2025)
- [Pareto Low-Rank Adapters: Efficient Multi-Task Learning with Preferences](https://mlsanity.com/paper/2407.08056/) (2024)
- [A Survey of Controllable Learning: Methods and Applications in Information Retrieval](https://mlsanity.com/paper/2407.06083/) (2024)
- [Efficient Pareto Manifold Learning with Low-Rank Structure](https://mlsanity.com/paper/2407.20734/) (2024)
- [COS-DPO: Conditioned One-Shot Multi-Objective Fine-Tuning Framework](https://mlsanity.com/paper/2410.08316/) (2024)
- [Multi-Objective Optimization via Wasserstein-Fisher-Rao Gradient Flow](https://mlsanity.com/paper/2311.13159/) (2023)
- [A Hyper-Transformer model for Controllable Pareto Front Learning with Split Feasibility Constraints](https://mlsanity.com/paper/2402.05955/) (2024)
- [Pareto Front Approximation for Multi-Objective Session-Based Recommender Systems](https://mlsanity.com/paper/2407.16828/) (2024)
- [Multi-objective Differentiable Neural Architecture Search](https://mlsanity.com/paper/2402.18213/) (2024)
- [Collaborative Pareto Set Learning in Multiple Multi-Objective Optimization Problems](https://mlsanity.com/paper/2404.01224/) (2024)
- [Data-Driven Preference Sampling for Pareto Front Learning](https://mlsanity.com/paper/2404.08397/) (2024)
- [The Fairness-Quality Trade-off in Clustering](https://mlsanity.com/paper/2408.10002/) (2024)
- [A Framework for Controllable Multi-objective Learning with Annealed Stein Variational Hypernetworks](https://mlsanity.com/paper/2506.06715/) (2025)
- [An approach to extract information from academic transcripts of HUST](https://mlsanity.com/paper/2304.11454/) (2023)
- [Multiobjective Logistics Optimization for Automated ATM Cash Replenishment Process](https://mlsanity.com/paper/2304.13671/) (2023)

## Abstract (arXiv)

Pareto Front Learning (PFL) was recently introduced as an effective approach to obtain a mapping function from a given trade-off vector to a solution on the Pareto front, which solves the multi-objective optimization (MOO) problem. Due to the inherent trade-off between conflicting objectives, PFL offers a flexible approach in many scenarios in which the decision makers can not specify the preference of one Pareto solution over another, and must switch between them depending on the situation. However, existing PFL methods ignore the relationship between the solutions during the optimization process, which hinders the quality of the obtained front. To overcome this issue, we propose a novel PFL framework namely PHN-HVI, which employs a hypernetwork to generate multiple solutions from a set of diverse trade-off preferences and enhance the quality of the Pareto front by maximizing the Hypervolume indicator defined by these solutions. The experimental results on several MOO machine learning tasks show that the proposed framework significantly outperforms the baselines in producing the trade-off Pareto front.