# Controllable Multi-Objective Re-ranking with Policy Hypernetworks

arXiv: https://arxiv.org/abs/2306.05118 · HTML: https://mlsanity.com/paper/2306.05118/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jun 2023.

## Built on by (most cited first, 15 of 15)

- [Bone Soups: A Seek-and-Soup Model Merging Approach for Controllable Multi-Objective Generation](https://mlsanity.com/paper/2502.10762/) (2025)
- [A Survey of Controllable Learning: Methods and Applications in Information Retrieval](https://mlsanity.com/paper/2407.06083/) (2024)
- [HyperBandit: Contextual Bandit with Hypernewtork for Time-Varying User Preferences in Streaming Recommendation](https://mlsanity.com/paper/2308.08497/) (2023)
- [Interactive Recommendation Agent with Active User Commands](https://mlsanity.com/paper/2509.21317/) (2025)
- [KuaiLive: A Real-time Interactive Dataset for Live Streaming Recommendation](https://mlsanity.com/paper/2508.05633/) (2025)
- [COS-DPO: Conditioned One-Shot Multi-Objective Fine-Tuning Framework](https://mlsanity.com/paper/2410.08316/) (2024)
- [A Hyper-Transformer model for Controllable Pareto Front Learning with Split Feasibility Constraints](https://mlsanity.com/paper/2402.05955/) (2024)
- [A Comprehensive Review on Harnessing Large Language Models to Overcome Recommender System Challenges](https://mlsanity.com/paper/2507.21117/) (2025)
- [UOEP: User-Oriented Exploration Policy for Enhancing Long-Term User Experiences in Recommender Systems](https://mlsanity.com/paper/2401.09034/) (2024)
- [Impression-Aware Recommender Systems](https://mlsanity.com/paper/2308.07857/) (2023)
- [Enhancing Bandit Algorithms with LLMs for Time-varying User Preferences in Streaming Recommendations](https://mlsanity.com/paper/2602.08067/) (2026)
- [GenRecEdit: Adapting Model Editing for Generative Recommendation with Cold-Start Items](https://mlsanity.com/paper/2603.14259/) (2026)
- [Transition-Matrix Regularization for Next Dialogue Act Prediction in Counselling Conversations](https://mlsanity.com/paper/2604.18539/) (2026)
- [Do Not Wait: Learning Re-Ranking Model Without User Feedback At Serving Time in E-Commerce](https://mlsanity.com/paper/2406.14004/) (2024)
- [A Production-Ready RL Framework for Personalized Utility Tuning with Pareto Sweeping in Pinterest Recommender Systems](https://mlsanity.com/paper/2605.16344/) (2026)

## Abstract (arXiv)

Multi-stage ranking pipelines have become widely used strategies in modern recommender systems, where the final stage aims to return a ranked list of items that balances a number of requirements such as user preference, diversity, novelty etc. Linear scalarization is arguably the most widely used technique to merge multiple requirements into one optimization objective, by summing up the requirements with certain preference weights. Existing final-stage ranking methods often adopt a static model where the preference weights are determined during offline training and kept unchanged during online serving. Whenever a modification of the preference weights is needed, the model has to be re-trained, which is time and resources inefficient. Meanwhile, the most appropriate weights may vary greatly for different groups of targeting users or at different time periods (e.g., during holiday promotions). In this paper, we propose a framework called controllable multi-objective re-ranking (CMR) which incorporates a hypernetwork to generate parameters for a re-ranking model according to different preference weights. In this way, CMR is enabled to adapt the preference weights according to the environment changes in an online manner, without retraining the models. Moreover, we classify practical business-oriented tasks into four main categories and seamlessly incorporate them in a new proposed re-ranking model based on an Actor-Evaluator framework, which serves as a reliable real-world testbed for CMR. Offline experiments based on the dataset collected from Taobao App showed that CMR improved several popular re-ranking models by using them as underlying models. Online A/B tests also demonstrated the effectiveness and trustworthiness of CMR.