# Interaction Dynamics as a Reward Signal for LLMs

arXiv: https://arxiv.org/abs/2511.08394 · HTML: https://mlsanity.com/paper/2511.08394/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 6 of its 9 references that are in the map.
- First posted on arXiv 11 Nov 2025.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Human values and AI feedback alignment](https://mlsanity.com/alignment/human-values-and-ai-feedback-alignment/)

## Builds on (most cited first, 6 of 6)

- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities](https://mlsanity.com/paper/2201.06796/) (2022)
- [Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionals](https://mlsanity.com/paper/2209.14958/) (2022)
- [Evaluating Human-AI Collaboration: A Review and Methodological Framework](https://mlsanity.com/paper/2407.19098/) (2024)
- [Writing as a testbed for open ended agents](https://mlsanity.com/paper/2503.19711/) (2025)
- [Opportunities and Challenges of LLMs in Education: An NLP Perspective](https://mlsanity.com/paper/2507.22753/) (2025)

## Built on by (most cited first, 5 of 5)

- [MT-PingEval: Evaluating Multi-Turn Collaboration with Private Information Games](https://mlsanity.com/paper/2602.24188/) (2026)
- [TopoCurate:Modeling Interaction Topology for Tool-Use Agent Training](https://mlsanity.com/paper/2603.01714/) (2026)
- [When Do We Need LLMs? A Diagnostic for Language-Driven Bandits](https://mlsanity.com/paper/2604.05859/) (2026)
- [PsychoPass: Geometric Profiling of Multi-Turn Adversarial LLM Conversations](https://mlsanity.com/paper/2606.03136/) (2026)
- [Speak for Me: Giving LLMs the Situational Awareness to Participate in a Meeting](https://mlsanity.com/paper/2609.03923/) (2026)

## Abstract (arXiv)

The alignment of Large Language Models (LLMs) for multi-turn conversations typically relies on reward signals derived from the content of the text. This approach, however, overlooks a rich, complementary source of signal: the dynamics of the interaction itself. This paper introduces TRACE (Trajectory-based Reward for Agent Collaboration Estimation), a novel reward signal derived from the geometric properties of a dialogue's embedding trajectory--a concept we term 'conversational geometry'. Our central finding is that a reward model trained only on these structural signals achieves a pairwise accuracy (68.20%) comparable to a powerful LLM baseline that analyzes the full transcript (70.04%). Furthermore, a hybrid model combining interaction dynamics with textual analysis achieves the highest performance (80.17%), demonstrating their complementary nature. This work provides strong evidence that for interactive settings, how an agent communicates is as powerful a predictor of success as what it says, offering a new, privacy-preserving framework that not only aligns agents but also serves as a diagnostic tool for understanding the distinct interaction patterns that drive successful collaboration.