# Enhancing Social Intelligence in LLMs with Hierarchical Reasoning and Utterance-Level Goal Rewarding

arXiv: https://arxiv.org/abs/2608.05832 · HTML: https://mlsanity.com/paper/2608.05832/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 17 of its 45 references that are in the map.
- First posted on arXiv 6 Aug 2026.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)

## Builds on (most cited first, 17 of 17)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [OpenAI o1 System Card](https://mlsanity.com/paper/2412.16720/) (2024)
- [DeepSeek-V3 Technical Report](https://mlsanity.com/paper/2412.19437/) (2024)
- [FeUdal Networks for Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1703.01161/) (2017)
- [Reinforcement Learning Optimization for Large-Scale Learning: An Efficient and User-Friendly Scaling Library](https://mlsanity.com/paper/2506.06122/) (2025)
- [ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2503.09501/) (2025)
- [Sotopia-RL: Reward Design for Social Intelligence](https://mlsanity.com/paper/2508.03905/) (2025)
- [SOTOPIA-Omega: Dynamic Strategy Injection Learning and Social Instruction Following Evaluation for Social Agents](https://mlsanity.com/paper/2502.15538/) (2025)
- [Adaptive Social Learning via Mode Policy Optimization for Language Agents](https://mlsanity.com/paper/2505.02156/) (2025)
- [Dialogue Action Tokens: Steering Language Models in Goal-Directed Dialogue with a Multi-Turn Planner](https://mlsanity.com/paper/2406.11978/) (2024)
- [The Imperative of Conversation Analysis in the Era of LLMs: A Survey of Tasks, Techniques, and Trends](https://mlsanity.com/paper/2409.14195/) (2024)
- [Debt Collection Negotiations with Large Language Models: An Evaluation System and Optimizing Decision Making with Multi-Agent](https://mlsanity.com/paper/2502.18228/) (2025)
- [EPO: Explicit Policy Optimization for Strategic Reasoning in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2502.12486/) (2025)
- [DEMO: Reframing Dialogue Interaction with Fine-grained Element Modeling](https://mlsanity.com/paper/2412.04905/) (2024)

## Abstract (arXiv)

Large language models (LLMs) excel in structured tasks but struggle with dynamic social interactions, where success requires long-term goal coordination and rapid adaptation. Current methods often apply uniform goal-based rewards to every utterance, overlooking the specificity of objectives at each dialogue turn and failing to account for the rationale of potential strategies. Inspired by the Theory of Planned Behavior, we propose the Think-Strategy-Response (TSR) framework, which decomposes social dialogue into two hierarchical stages: high-level strategic planning and low-level linguistic execution. To optimize TSR, we introduce Linearized Hierarchical Reinforcement Learning with Variance-Gated Rewards (LHRL-VGR), a novel algorithm that dynamically routes rewards - balancing goal completion and strategy adherence - based on the variance of goal achievement scores. Experiments on the SOTOPIA benchmark show that our approach fine-tunes a Qwen2.5-7B agent to surpass the GPT-4o baseline by 7.32% in goal completion success, demonstrating state-of-the-art performance in multi-agent social negotiation tasks.