# Models Recall What They Violate: Constraint Adherence in Multi-Turn LLM Ideation

arXiv: https://arxiv.org/abs/2604.28031 · HTML: https://mlsanity.com/paper/2604.28031/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 23 references that are in the map.
- First posted on arXiv 30 Apr 2026.

## Builds on (most cited first, 22 of 22)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [AgentBench: Evaluating LLMs as Agents](https://mlsanity.com/paper/2308.03688/) (2023)
- [LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory](https://mlsanity.com/paper/2410.10813/) (2024)
- [LLMs Get Lost In Multi-Turn Conversation](https://mlsanity.com/paper/2505.06120/) (2025)
- [AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents](https://mlsanity.com/paper/2401.13178/) (2024)
- [MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback](https://mlsanity.com/paper/2309.10691/) (2023)
- [ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities](https://mlsanity.com/paper/2408.04682/) (2024)
- [MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialogues](https://mlsanity.com/paper/2402.14762/) (2024)
- [MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMs](https://mlsanity.com/paper/2501.17399/) (2025)
- [DISCOVERYWORLD: A Virtual Environment for Developing and Evaluating Automated Scientific Discovery Agents](https://mlsanity.com/paper/2406.06769/) (2024)
- [MT-Eval: A Multi-Turn Capabilities Evaluation Benchmark for Large Language Models](https://mlsanity.com/paper/2401.16745/) (2024)
- [Beyond Single-Turn: A Survey on Multi-Turn Interactions with Large Language Models](https://mlsanity.com/paper/2504.04717/) (2025)
- [HypoBench: Towards Systematic and Principled Benchmarking for Hypothesis Generation](https://mlsanity.com/paper/2504.11524/) (2025)
- [CoQuest: Exploring Research Question Co-Creation with an LLM-based Agent](https://mlsanity.com/paper/2310.06155/) (2023)
- [Evaluating LLMs' Divergent Thinking Capabilities for Scientific Idea Generation with Minimal Context](https://mlsanity.com/paper/2412.17596/) (2024)
- [Drift No More? Context Equilibria in Multi-Turn LLM Interactions](https://mlsanity.com/paper/2510.07777/) (2025)
- [PersonaFlow: Designing LLM-Simulated Expert Perspectives for Enhanced Research Ideation](https://mlsanity.com/paper/2409.12538/) (2024)
- [TurnBench-MS: A Benchmark for Evaluating Multi-Turn, Multi-Step Reasoning in Large Language Models](https://mlsanity.com/paper/2506.01341/) (2025)
- [TRACE: Trajectory-Aware Comprehensive Evaluation for Deep Research Agents](https://mlsanity.com/paper/2602.21230/) (2026)
- [Evaluating Performance Drift from Model Switching in Multi-Turn LLM Systems](https://mlsanity.com/paper/2603.03111/) (2026)

## Built on by (most cited first, 3 of 3)

- [MAIGO: Mitigating Lost-in-Conversation with History-Cleaned On-Policy Self-Distillation](https://mlsanity.com/paper/2605.27186/) (2026)
- [Dead text or binding clause? Measuring and restoring constraint influence in black-box LLM dialogues](https://mlsanity.com/paper/2608.12599/) (2026)
- [Visible but Not Avoided: State-Dependent Re-Proposal of Rejected Candidates in Catalog-Based LLM Optimization Loops](https://mlsanity.com/paper/2610.15738/) (2026)

## Abstract (arXiv)

When researchers iteratively refine ideas with large language models, do the models preserve fidelity to the original objective? We introduce DriftBench, a benchmark for evaluating constraint adherence in multi-turn LLM-assisted scientific ideation. Across 2,146 scored benchmark runs spanning seven models from five providers (including two open-weight), four interaction conditions, and 38 research briefs from 24 scientific domains, we find that iterative pressure reliably increases structural complexity and often reduces adherence to original constraints. A restatement probe reveals a dissociation between declarative recall and behavioral adherence, as models accurately restate constraints they simultaneously violate. The knows-but-violates (KBV) rate, measuring constraint non-compliance despite preserved recall, ranges from 8% to 99% across models. Structured checkpointing partially reduces KBV rates but does not close the dissociation, and complexity inflation persists. Human validation against blind raters confirms that the LLM judge under-detects constraint violations, making reported constraint adherence scores conservative. Sensitivity analyses confirm the findings are robust to temperature (0.7 vs. 1.0) and pressure type (novelty vs. rigor). We release all briefs, prompts, rubrics, transcripts, and scores as an open benchmark.