mlsanityData to 13 Oct 2026

Teaching LLM to be Persuasive: Reward-Enhanced Policy Optimization for Alignment from Heterogeneous Rewards

arXiv 2510.04214 ↗First posted 5 Oct 2025cs.CLcited by 1

On the map: LLM alignment / RLHF / preference optimisation → Reward modeling and RLHF

Abstract (arXiv)

We deploy large language models (LLMs) as business development (BD) agents for persuasive price negotiation in online travel agencies (OTAs). The agent must follow a multi-stage Standard Operating Procedure (SOP) and strict guardrails (no over-promising and no hallucinations), while remaining human-like and effective over long, multi-turn dialogues. We propose Reward-Enhanced Policy Optimization (REPO), a reinforcement learning post-training method that combines heterogeneous rewards: a preference-trained reward model (RM), an LLM-as-a-judge (RJ) for nuanced behaviors (e.g., emotional value and SOP compliance), and rule-based reward functions (RF) (mainly regex-based) for deterministic checks on numerics, formatting, and guardrails. In expert consensus evaluation (three human experts; 30 online conversations and 45 curated bad cases), REPO improves average dialogue rating to 4.63 (+0.33 over GRPO) and raises the share of conversations with at least one excellent response to 66.67% (+23.34 pp over GRPO), while achieving a 93.33% bad-case fix rate with 75.56% clean fixes. In a production A/B test on 9,653 real customer conversations (vs. an intent-driven dialogue system), REPO improves response rate by +12.14 pp and task success rate by +5.94 pp (p<0.001).

Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 8 of its 75 references that are in the map. First posted on arXiv 5 Oct 2025.

Builds on

8 of 75 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Agentic Reward Modeling: Integrating Human Preferences with Verifiable Correctness Signals for Reliable Reward Systems26 Feb
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Model29 May

Other references · 6

  1. 2022Training language models to follow instructions with human feedbackcited by 15,856
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,231
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,091
  4. 2024A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systemscited by 170
  5. 2024MORL-Prompt: An Empirical Analysis of Multi-Objective Reinforcement Learning for Discrete Prompt Optimizationcited by 16
  6. 2025Empowering LLMs in Task-Oriented Dialogues: A Domain-Independent Multi-Agent Framework and Fine-Tuning Strategycited by 3

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026DIPLOMAT: Dialogue-Span-Aware Direct Preference Optimization for Polite Persuasive Workplace Negotiation Dialoguescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Two Minds Better Than One: Collaborative Reward Modeling for LLM Alignment
  2. 2024Reward-Augmented Data Enhances Direct Preference Alignment of LLMs
  3. 2024T-REG: Preference Optimization with Token-Level Reward Regularization
  4. 2025Uncertainty Quantification for Large Language Model Reward Learning under Heterogeneous Human Feedback
  5. 2025Mutual-Taught for Co-adapting Policy and Reward Models
  6. 2024Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferences
  7. 2026Do Small Language Models Learn to Negotiate? A Controlled Scaling Study of RL-Trained Sellers
  8. 2024Exploring RL-based LLM Training for Formal Language Tasks with Programmed Rewards