mlsanityData to 7 Oct 2026

Balancing Multiple Objectives in Urban Traffic Control with Reinforcement Learning from AI Feedback

arXiv 2602.20728 ↗First posted 24 Feb 2026cs.AIcited by 1

On the map: LLM alignment / RLHF / preference optimisation → Reward modeling and RLHF

Abstract (arXiv)

Reward design has been one of the central challenges for real world reinforcement learning (RL) deployment, especially in settings with multiple objectives. Preference-based RL offers an appealing alternative by learning from human preferences over pairs of behavioural outcomes. More recently, RL from AI feedback (RLAIF) has demonstrated that large language models (LLMs) can generate preference labels at scale, mitigating the reliance on human annotators. However, existing RLAIF work typically focuses only on single-objective tasks, leaving the open question of how RLAIF handles systems that involve multiple objectives. In such systems trade-offs among conflicting objectives are difficult to specify, and policies risk collapsing into optimizing for a dominant goal. In this paper, we explore the extension of the RLAIF paradigm to multi-objective self-adaptive systems. We show that multi-objective RLAIF can produce policies that yield balanced trade-offs reflecting different user priorities without laborious reward engineering. We argue that integrating RLAIF into multi-objective RL offers a scalable path toward user-aligned policy learning in domains with inherently conflicting objectives.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 11 of its 26 references that are in the map. First posted on arXiv 24 Feb 2026.

Builds on

11 of 26 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025TREND: Tri-teaching for Robust Preference-based Reinforcement Learning with Demonstrations9 May
  2. 2023Text2Reward: Reward Shaping with Language Models for Reinforcement Learning20 Sep
  3. 2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training9 Jun
  4. 2017Deep reinforcement learning from human preferences12 Jun

Other references · 7

  1. 2024RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedbackcited by 147
  2. 2021B-Pref: Benchmarking Preference-Based Reinforcement Learningcited by 106
  3. 2023Motif: Intrinsic Motivation from Artificial Intelligence Feedbackcited by 99
  4. 2023Read and Reap the Rewards: Learning to Play Atari with the Help of Instruction Manualscited by 45
  5. 2019Diagnosing Reinforcement Learning for Traffic Signal Controlcited by 33
  6. 2017Adaptive Traffic Signal Control: Deep Reinforcement Learning Algorithm with Experience Replay and Target Networkcited by 16
  7. 2023Deep Reinforcement Learning from Hierarchical Preference Designcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Distributed Zeroth-Order Policy Gradient for Networked Multi-agent Reinforcement Learning from Human Feedbackcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Multi-objective Reinforcement learning from AI Feedback
  2. 2026LEMUR: Learning to Align with Multi-Objective Reinforcement Learning from Preference Feedback
  3. 2026Reinforcement Learning from Human Feedback: A Statistical Perspective
  4. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback
  5. 2025MPO: An Efficient Post-Processing Framework for Mixing Diverse Preference Alignment
  6. 2025A Technical Survey of Reinforcement Learning Techniques for Large Language Models
  7. 2025Preference-based Multi-Objective Reinforcement Learning
  8. 2023Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-Constraint