Rolling With Resistance: Preference-Optimized LLM Counselors Can Trade Goal Persistence for Relational Attunement in Motivational Interviewing
Not on the map. Why?
Abstract (arXiv)
In Motivational Interviewing (MI), a client's sustain talk (arguments for the status quo) calls for the counselor to roll with resistance, a move that can fail in two opposite ways: capitulation (abandoning the change agenda to preserve rapport) or confrontation (arguing or directing, overriding the client's autonomy). We introduce a two-axis evaluation of counselor responses, anchored in the Motivational Interviewing Treatment Integrity (MITI) code, Goal Persistence (GP) and Relational Attunement (RA), yielding a four-quadrant framing in which rolling with resistance is high on both, and we ask whether penalizing one failure through preference optimization teaches rolling with resistance or provokes its opposite. From the expert-annotated AnnoMI corpus we build topic-disjoint Direct Preference Optimization data whose preference sets differ only in which failure is rejected, using on-policy negatives. An automatic judge, validated against AnnoMI's expert labels and rechecked by trained human coders, scores blind pairwise win-rates against each base under a firewall in which disjoint model families generate, label, and judge. Across three aligned instruction models spanning the Qwen and Llama families, penalizing confrontation reliably lowers goal persistence below parity, on every base and in every seed run, a robust cost, whereas the attunement gain is base-dependent, present on two of the three bases but absent on the third. Penalizing capitulation is inert, because these models rarely capitulate on-policy, so the trade is gated by each base's failure profile. A prompt-only control raises attunement without the goal-persistence cost, locating the cost in the optimization rather than in attunement itself.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 27 references that are in the map. First posted on arXiv 30 Jul 2026.
Builds on
15 of 27 references.
References · 15
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
- 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
- 2023Large Language Models are not Fair Evaluatorscited by 1,050
- 2023Towards Understanding Sycophancy in Language Modelscited by 988
- 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
- 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 651
- 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Trainingcited by 399
- 2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 232
- 2024Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewardscited by 145
- 2024Language Models Learn to Mislead Humans via RLHFcited by 118
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Beyond Compliance: A Resistance-Informed Motivation Reasoning Framework for Challenging Psychological Client Simulation
- 2026MIThinker: A Plug-and-Play Policy-Optimized Thinker For Motivational Interviewing Counseling
- 2025CAMI: A Counselor Agent Supporting Motivational Interviewing through State Inference and Topic Exploration
- 2026Preference Tree Optimization: Enhancing Goal-Oriented Dialogue with Look-Ahead Simulations
- 2026CALM-IT: Generating Realistic Long-Form Motivational Interviewing Dialogues with Dual-Actor Conversational Dynamics Tracking
- 2025Preference Learning Unlocks LLMs' Psycho-Counseling Skills
- 2026When Do LLM Preferences Predict Downstream Behavior?
- 2024Diverging Preferences: When do Annotators Disagree and do Models Know?