# Entropy in Conversational AI: Structured Unpredictability as Inferrable Interiority

arXiv: https://arxiv.org/abs/2609.19044 · HTML: https://mlsanity.com/paper/2609.19044/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 33 references that are in the map.
- First posted on arXiv 20 Jul 2026.

## Builds on (most cited first, 24 of 24)

- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018)
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://mlsanity.com/paper/1912.02164/) (2019)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [Contrastive Decoding: Open-ended Text Generation as Optimization](https://mlsanity.com/paper/2210.15097/) (2022)
- [Texygen: A Benchmarking Platform for Text Generation Models](https://mlsanity.com/paper/1802.01886/) (2018)
- [GeDi: Generative Discriminator Guided Sequence Generation](https://mlsanity.com/paper/2009.06367/) (2020)
- [FUDGE: Controlled Text Generation With Future Discriminators](https://mlsanity.com/paper/2104.05218/) (2021)
- [Character-LLM: A Trainable Agent for Role-Playing](https://mlsanity.com/paper/2310.10158/) (2023)
- [Role-Play with Large Language Models](https://mlsanity.com/paper/2305.16367/) (2023)
- [What makes a good conversation? How controllable attributes affect human judgments](https://mlsanity.com/paper/1902.08654/) (2019)
- [Language GANs Falling Short](https://mlsanity.com/paper/1811.02549/) (2018)
- [Same Author or Just Same Topic? Towards Content-Independent Style Representations](https://mlsanity.com/paper/2204.04907/) (2022)
- [Measuring and Controlling Instruction (In)Stability in Language Model Dialogs](https://mlsanity.com/paper/2402.10962/) (2024)
- [PersonaAgent: Bridging Memory and Action for Personalized LLM Agents](https://mlsanity.com/paper/2506.06254/) (2025)
- [SPASM: Stable Persona-driven Agent Simulation for Multi-turn Dialogue Generation](https://mlsanity.com/paper/2604.09212/) (2026)
- [Echoing: Identity Failures when LLM Agents Talk to Each Other](https://mlsanity.com/paper/2511.09710/) (2025)
- [ContextEcho: A Benchmark for Persona Drift in Long Agentic-Coding Sessions](https://mlsanity.com/paper/2605.24279/) (2026)
- [Scaling Self-Evolving Agents via Parametric Memory](https://mlsanity.com/paper/2606.04536/) (2026)
- [Perceived AGI: Believability as Dimensional Completeness, Not Capability](https://mlsanity.com/paper/2607.15883/) (2026)

## Built on by (most cited first, 1 of 1)

- [Truth for Believable AI: Expressed Doubt, Provenance, and Belief Revision as an Engineerable Stance](https://mlsanity.com/paper/2609.26035/) (2026)

## Abstract (arXiv)

Sampling can increase response diversity without producing history-dependent behavior. We formalize a different design target, structured unpredictability, as conditional dependence between an output and a persistent hidden state beyond what an observer can infer from the transcript. A selection layer updates a low-dimensional style-and-attention state from a capacity-limited stream, generates several responses with a fixed base model, and selects for novelty and state affinity. Evaluation uses scripted sequences of independent prompt turns: the base model receives the current turn and rendered state, but not the preceding dialogue; cross-turn dependence resides in the wrapper state and response selector. A synthetic implementation validates the pipeline and matches four prospectively hash-frozen divergence features at point level. In the final real-model grid (mlx-community/Qwen2.5-1.5B-Instruct-4bit; 56 sequences per arm), the mechanism increased lexical novelty over the low-variance and consistency-only controls by 0.073 and 0.023, respectively. Its stylometric-consistency contrast with novelty-matched sampling was equivalent to zero under the registered smallest-effect rule, so the joint novelty-consistency criterion failed. The original two-part accumulation criterion also failed; a revised final-grid contrast, frozen after the powered grid, found higher consistency than the memory-reset ablation (0.028, 95% CI [0.018,0.039]), but does not establish path dependence. Twin separation was not established (0.003, 95% CI [-0.011,0.019]); the mean curve's saturating curvature matched the frozen prediction, which without separation does not support path dependence. Probe-level capability equivalence held within +/-0.10 on a near-ceiling battery, while output quality was not evaluated. All outcomes are machine-scored; no claims about perceived mind or consciousness are tested.