# Controlling Long-Horizon Behavior in Language Model Agents with Explicit State Dynamics

arXiv: https://arxiv.org/abs/2601.16087 · HTML: https://mlsanity.com/paper/2601.16087/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 9 of its 19 references that are in the map.
- First posted on arXiv 22 Jan 2026.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 9 of 9)

- [Hello Again! LLM-powered Personalized Agent for Long-term Dialogue](https://mlsanity.com/paper/2406.05925/) (2024)
- [HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Model](https://mlsanity.com/paper/2408.09559/) (2024)
- [Prompted LLMs as Chatbot Modules for Long Open-domain Conversation](https://mlsanity.com/paper/2305.04533/) (2023)
- [Augmenting Transformers with KNN-Based Composite Memory for Dialogue](https://mlsanity.com/paper/2004.12744/) (2020)
- [Building a Role Specified Open-Domain Dialogue System Leveraging Large-Scale Language Models](https://mlsanity.com/paper/2205.00176/) (2022)
- [AnnaAgent: Dynamic Evolution Agent System with Multi-Session Memory for Realistic Seeker Simulation](https://mlsanity.com/paper/2506.00551/) (2025)
- [Large Scale Multi-Actor Generative Dialog Modeling](https://mlsanity.com/paper/2005.06114/) (2020)
- [UniMC: A Unified Framework for Long-Term Memory Conversation via Relevance Representation Learning](https://mlsanity.com/paper/2306.10543/) (2023)
- [PersLLM: A Personified Training Approach for Large Language Models](https://mlsanity.com/paper/2407.12393/) (2024)

## Built on by (most cited first, 3 of 3)

- [Agentic Trading: When LLM Agents Meet Financial Markets](https://mlsanity.com/paper/2605.19337/) (2026)
- [How Emotion Shapes the Behavior of LLMs and Agents: A Mechanistic Study](https://mlsanity.com/paper/2604.00005/) (2026)
- [Modeling Clinical Concern Trajectories in Language Model Agents](https://mlsanity.com/paper/2604.27872/) (2026)

## Abstract (arXiv)

Large language model (LLM) agents often exhibit abrupt shifts in tone and persona during extended interaction, reflecting the absence of explicit temporal structure governing agent-level state. While prior work emphasizes turn-local sentiment or static emotion classification, the role of explicit affective dynamics in shaping long-horizon agent behavior remains underexplored. This work investigates whether imposing dynamical structure on an external affective state can induce temporal coherence and controlled recovery in multi-turn dialogue. We introduce an agent-level affective subsystem that maintains a continuous Valence-Arousal-Dominance (VAD) state external to the language model and governed by first- and second-order update rules. Instantaneous affective signals are extracted using a fixed, memoryless estimator and integrated over time via exponential smoothing or momentum-based dynamics. The resulting affective state is injected back into generation without modifying model parameters. Using a fixed 25-turn dialogue protocol, we compare stateless, first-order, and second-order affective dynamics. Stateless agents fail to exhibit coherent trajectories or recovery, while state persistence enables delayed responses and reliable recovery. Second-order dynamics introduce affective inertia and hysteresis that increase with momentum, revealing a trade-off between stability and responsiveness.