mlsanityData to 14 Oct 2026

Evaluation of Multi-Turn Consistency in LLM Agents: Survival Analysis and Failure-Rationale Taxonomy

arXiv 2609.29508 ↗First posted 26 Aug 2026cs.AI cs.CL cs.LGcited by 1

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Large language model (LLM) agents may perform well on isolated tasks yet drift into inconsistency over extended interaction. We evaluate temporal consistency in a controlled 20-step multi-agent setting inspired by delayed-gratification studies. At each step, an agent chooses between continuing to delay a reward or claiming it immediately (terminating the episode). Across a full-factorial manipulation of social visibility (private vs public), persona stressors, and deliberation policy, we run 84,540 trajectories spanning 8 model families. Treating the first reward-claim as a time-to-event outcome, we estimate Kaplan-Meier survival curves and fit discrete-time hazard regression to quantify how experimental factors shift failure risk over time. Then, to analyze rationales and language patterns associated with failure, we build a seven-category taxonomy from 13,780 deliberation traces from agents who choose to terminate the episode, using an LLM-assisted labeling paired with human audit (kappa=0.83). Rationale profiles change systematically with time and context: early failures are more impulse-driven, later failures more fatigue- and cost-benefit-framed, while public settings increase norm-oriented justifications. We also find a deliberation-inconsistency association: among failures, longer deliberation correlates with higher rates of intra-rationale contradiction (simultaneous pro-delay and pro-claim statements), challenging the assumption that more reasoning text implies greater consistency. Together, the survival and rationale analyses reveal distinct temporal reliability regimes and model-specific "failure fingerprints", offering an evaluation lens for diagnosing inconsistency in multi-turn agent behavior.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 15 of its 32 references that are in the map. First posted on arXiv 26 Aug 2026.

Builds on

15 of 32 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents24 Jan
  2. 2023AgentBench: Evaluating LLMs as Agents7 Aug

Other references · 13

  1. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  2. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  3. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  4. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 810
  5. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
  6. 2025LLMs Get Lost In Multi-Turn Conversationcited by 429
  7. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 320
  8. 2023MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedbackcited by 297
  9. 2022Thinking Fast and Slow in Large Language Modelscited by 137
  10. 2025Evaluating Step-by-step Reasoning Traces: A Surveycited by 63
  11. 2024LLM economicus? Mapping the Behavioral Biases of LLMs via Utility Theorycited by 47
  12. 2025Measuring Chain of Thought Faithfulness by Unlearning Reasoning Stepscited by 47
1 more
  1. 2023A Closer Look at the Self-Verification Abilities of Large Language Models in Logical Reasoningcited by 41

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Delay-of-Gratification as a Multi-Agent Survival Micro-benchmark for Long-Horizon LLMs: Social Exposure, Personas, and Tool Use Budgets
  2. 2026Beyond the Leaderboard: A Synthesis of Tool-Use, Planning, and Reasoning Failures in Large Language Model Agents
  3. 2026ReLiveGym: Evaluating Long-Lived Agents over Weeks of Replayed Reality
  4. 2026Survive at All Costs: Exploring LLM's Risky Behaviors under Survival Pressure
  5. 2026Silent Failure in LLM Agent Systems: The Entropy Principle and the Inevitable Disorder of Autonomous Agents
  6. 2025Check Yourself Before You Wreck Yourself: Selectively Quitting Improves LLM Agent Safety
  7. 2026Agent Drift: Quantifying Behavioral Degradation in Multi-Agent LLM Systems Over Extended Interactions
  8. 2025How Do LLMs Fail In Agentic Scenarios? A Qualitative Analysis of Success and Failure Scenarios of Various LLMs in Agentic Simulations