# Quantifying Conversational Reliability of Large Language Models under Multi-Turn Interaction

arXiv: https://arxiv.org/abs/2603.01423 · HTML: https://mlsanity.com/paper/2603.01423/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 11 references that are in the map.
- First posted on arXiv 2 Mar 2026.

## Builds on (most cited first, 3 of 3)

- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback](https://mlsanity.com/paper/2309.10691/) (2023)
- [StructFlowBench: A Structured Flow Benchmark for Multi-turn Instruction Following](https://mlsanity.com/paper/2502.14494/) (2025)

## Built on by (most cited first, 1 of 1)

- [Evaluating Epistemic Guardrails in AI Reading Assistants: A Behavioral Audit of a Minimal Prototype](https://mlsanity.com/paper/2604.27275/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are increasingly deployed in real-world applications where users engage in extended, mixed-topic conversations that depend on prior context. Yet, their reliability under realistic multi-turn interactions remains poorly understood. We conduct a systematic evaluation of conversational reliability through three representative tasks that reflect practical interaction challenges: (1) maintaining global constraints across topic shifts, (2) selecting the correct tool or agent amid interleaved intents, and (3) tracking structured entities under revisions and distractions. Each task pairs single-turn and multi-turn settings, allowing us to quantify reliability degradation under extended dialogue. Across both commercial and open-source models, we observe substantial declines in reliability, particularly for smaller models. Error analyses reveal recurring failure modes such as instruction drift, intent confusion, and contextual overwriting, which compromise dependable behavior in operational systems. Our findings highlight the need for stress-testing LLMs for conversational reliability and developing more robust evaluation methods for trustworthy deployment.