mlsanityData to 14 Oct 2026

Measuring and Improving Behavioral Consistency in Large Language Models through Fact-Heuristic-Emotion State Enforcement

arXiv 2607.24765 ↗First posted 5 Jun 2026cs.CL cs.AI cs.HCcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) can give different answers to the same decision problem across runs, and reverse a decision when their own prior answer returns as context. We ask whether this instability can be measured and partially reduced without changing model weights. We test the Cognitive Kernel Model (CKM), a prompt-level state-enforcement layer. Before deciding, the model must separate its input into three epistemic roles: Fact (given or verifiable), Heuristic (inferred or assumed), and Emotion (evaluative or priority signal). CKM adds no capability; it forces the model to track what kind of information it uses before acting. Formally it maintains a structured state S_t = {F_t, H_t, E_t} updated by a transition function. We evaluate CKM on Korean-language decision scenarios (ambiguity, ethical conflict, resource allocation, error handling) across 26 LLMs from four vendors and 37,403 observations, via four core experiments, a 4-arm ablation, a 5-arm sham-restriction ablation, and a temperature probe. Findings: (1) CKM reduces repeated-output variability (random-effects Hedges' g=1.09, 95% CI [0.83, 1.35], 31 model pairs); (2) state persistence cuts the decision-flip rate by 82% in newer models (g=1.52); (3) the effect is not JSON formatting alone (value-only recomputation, g=2.24); (4) intrinsic randomness under fixed anchor states is negligible; (5) the advantage grows under sampling stochasticity (g=2.87 at temperature 0.7); (6) a sham ablation attributes about 45% of the gain to structural scaffolding and 55% to Fact/Heuristic/Emotion content, and CKM is the only arm that both raises consistency and reduces flipping. CKM does not improve reasoning correctness. The narrower result: behavioral consistency is measurable, varies across models, and is partially improvable by forcing models to separate facts, assumptions, and evaluative signals before deciding.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 32 of its 86 references that are in the map. First posted on arXiv 5 Jun 2026.

Builds on

32 of 86 references.

References · 32

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  6. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  7. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  8. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  9. 2025A-MEM: Agentic Memory for LLM Agentscited by 1,063
  10. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 886
  11. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
  12. 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 779
20 more
  1. 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 486
  2. 2023Cognitive Architectures for Language Agentscited by 429
  3. 2020Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performancecited by 399
  4. 2018On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detectioncited by 159
  5. 2025Memory OS of AI Agentcited by 150
  6. 2025Cognitive Foundations for Reasoning and Their Manifestation in LLMscited by 26
  7. 2025Assessing Consistency and Reproducibility in the Outputs of Large Language Models: Evidence Across Diverse Finance and Accounting Taskscited by 20
  8. 2025SCORE: Systematic COnsistency and Robustness Evaluation for Large Language Modelscited by 19
  9. 2025An overview of model uncertainty and variability in LLM-based sentiment analysis. Challenges, mitigation strategies and the role of explainabilitycited by 15
  10. 2025Git Context Controller: Manage the Context of LLM-based Agents like Gitcited by 15
  11. 2025Memoria: A Scalable Agentic Memory Framework for Personalized Conversational AIcited by 12
  12. 2026SPASM: Stable Persona-driven Agent Simulation for Multi-turn Dialogue Generationcited by 11
  13. 2024Cognitive Kernel: An Open-source Agent System towards Generalist Autopilotscited by 10
  14. 2025On the Failure of Latent State Persistence in Large Language Modelscited by 9
  15. 2025Behavioral Fingerprinting of Large Language Modelscited by 5
  16. 2026Behavioral Fingerprints for LLM Endpoint Stability and Identitycited by 4
  17. 2026PICon: A Multi-Turn Interrogation Framework for Evaluating Persona Agent Consistencycited by 3
  18. 2025From Stochasticity to Signal: A Bayesian Latent State Model for Reliable Measurement with LLMscited by 2
  19. 2026Enhancing Persona Following at Decoding Time via Dynamic Importance Estimation for Role-Playing Agentscited by 2
  20. 2025CAT: A Metric-Driven Framework for Analyzing the Consistency-Accuracy Relation of LLMs under Controlled Input Variationscited by 1

Built on by

Not cited yet.