# RELATE: An Evaluation Framework for measuring Relational Orientation of Large Language Models

arXiv: https://arxiv.org/abs/2610.09569 · HTML: https://mlsanity.com/paper/2610.09569/

## Key facts

- No citing papers in the map yet (data through 7 Oct 2026).
- Builds on 15 of its 41 references that are in the map.
- First posted on arXiv 7 Oct 2026.

## Builds on (most cited first, 15 of 15)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [DeepSeek-V3 Technical Report](https://mlsanity.com/paper/2412.19437/) (2024)
- [ResearchRubrics: A Benchmark of Prompts and Rubrics For Evaluating Deep Research Agents](https://mlsanity.com/paper/2511.07685/) (2025)
- [Investigating Affective Use and Emotional Well-being on ChatGPT](https://mlsanity.com/paper/2504.03888/) (2025)
- [How AI and Human Behaviors Shape Psychosocial Effects of Extended Chatbot Use: A Longitudinal Randomized Controlled Study](https://mlsanity.com/paper/2503.17473/) (2025)
- [HEART: A Unified Benchmark for Assessing Humans and LLMs in Emotional Support Dialogue](https://mlsanity.com/paper/2601.19922/) (2026)
- [Beyond Cooperative Simulators: Generating Realistic User Personas for Robust Evaluation of LLM Agents](https://mlsanity.com/paper/2605.12894/) (2026)
- [VERA-MH Concept Paper](https://mlsanity.com/paper/2510.15297/) (2025)
- [Relationship-Centered Care: Relatedness and Responsible Design for Human Connections in Mental-Health Care](https://mlsanity.com/paper/2603.18375/) (2026)
- [Auditing Support Strategies in LLMs through Grounded Multi-Turn Social Simulation](https://mlsanity.com/paper/2604.17079/) (2026)
- [CompanionBench: A Theory-Anchored, Real-World-Grounded Benchmark for AI Emotional Companionship](https://mlsanity.com/paper/2608.02046/) (2026)
- [Enjoy Your Talk: A Human-Centered Benchmark for Multi-Turn Dialogue with Decoupled User Simulation, Target Modeling, and Judging](https://mlsanity.com/paper/2607.10428/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are increasingly used for emotional support, raising concern that sustained use may draw users away from their real-world relationships. Yet existing evaluations primarily focus on the safety, empathy, or helpfulness of responses, leaving under-examined a relational question: where does the model orient the user for continued support? To address this question, we introduce relational orientation, a property operationalized through two non-exclusive dimensions: inward-facing (IF) language, which positions the AI as the user's ongoing source of support, and outward-scaffolding (OS) language, which encourages real-world human connection. Grounded in psychological and sociological literature, we formalize a taxonomy of relational orientation and present RELATE, a persona-conditioned framework for measuring inward-facing and outward-scaffolding language at the sentence level in multi-turn dialogues. RELATE pairs 76 help-seeking situations adapted from naturally occurring questions with three simulated user styles, providing 228 evaluation stimuli. In our experiments, we evaluate seven LLMs using dialogues with six assistant turns each, yielding 1,596 dialogues and 69,194 assistant sentences. We assess these sentences using a primary rubric-based LLM judge and apply a secondary judge to a subset. Under automated evaluation, we find that the proportion of sentences labeled as IF is higher at the sixth assistant turn than at the first, while the proportion labeled as OS is substantially lower for hesitant, indirect simulated users than for explicit, reassurance-seeking users. RELATE provides a reproducible framework and a sentence-level signal for auditing and steering the relational orientation of supportive LLMs.