Trivium: Temporal Regret as a First-Class Objective for Causal-Memory Controllers
On the map: LLM agents and tool use → Agent memory systems
Abstract (arXiv)
Many agentic systems and LLM pipelines correct mistakes by optimizing outcome reward. This addresses only the what of failure; the why and when may go unlogged, allowing the same error to recur across episodes. We propose long-horizon temporal regret alongside outcome regret and epistemic regret. These are diagnostic quantities, not standard comparator-based online-learning regrets. Temporal regret captures how long an unresolved or incorrect causal model is tolerated; epistemic regret captures posterior error over that model. Over a stream of E episodes, we prove three conditional results under explicit probing, persistence, and detectability assumptions. First, under observationally equivalent confounding, outcome-only learning cannot separate causal from spurious structure, so miscalibration can persist after outcome regret reaches zero. Second, with a persistent causal log and budgeted probes, total probe complexity is logarithmic in E, inducing O(log E) delayed-identification temporal regret. The implemented clipped soft score adds a linear numerical floor, so this logarithmic claim applies only to identification delay. Third, under K detectable change-points, the rate extends to O(K log E). We instantiate Trivium and state five falsifiable predictions. On CausalBench-Seq, a hard structural readout records 7.8+-2.9 misidentified episodes per seed over 500 episodes and 20 seeds, with zero observed stationary errors, while outcome-only controllers remain misidentified throughout. Audit ablations show that continued posterior updating, not detector reopening or local repair, drives posterior recovery; local repair instead reduces committed-graph dispatch exposure. The prior logarithmic-envelope verdict based on a clipped soft score is withdrawn. A pilot real-LLM stream provides external evidence. Self-learning here means revising an external causal model, not retraining LLM weights.
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. Builds on 9 of its 20 references that are in the map. First posted on arXiv 3 Jun 2026.
Builds on
9 of 20 references.
References · 9
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
- 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
- 2023Causal Reasoning and Large Language Models: Opening a New Frontier for Causalitycited by 353
- 2018Confounding-Robust Policy Improvementcited by 76
- 2020Off-policy Policy Evaluation For Sequential Decisions Under Unobserved Confoundingcited by 61
- 2025SagaLLM: Context Management, Validation, and Transaction Guarantees for Multi-Agent LLM Planningcited by 60
- 2025REALM-Bench: A Benchmark for Evaluating Multi-Agent Systems on Real-world, Dynamic Planning and Scheduling Taskscited by 16
- 2026Diagnosing and Mitigating Sycophancy and Skepticism in LLM Causal Judgmentcited by 5
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026Mnemosyne: Agentic Transaction Processing for Validating and Repairing AI-generated Workflowscited by 8
- 2026Architectural Wisdom: A Framework for Governing Optimization in AI Systemscited by 0
- 2026TRACE: An Operational Reasoning Schema for Auditable Agentic Commitmentscited by 0
- 2026TRW: TRACE-RealWorld---An Auditable Consistency Contract for World Models as Materialized Viewscited by 0
- 2026TRW: TRACE-RealWorld---An Auditable Consistency Contract for World Models as Materialized Views24 Jul 2026
- 2026TRACE: An Operational Reasoning Schema for Auditable Agentic Commitments14 Jul 2026
- 2026Mnemosyne: Agentic Transaction Processing for Validating and Repairing AI-generated Workflows30 Jun 2026
- 2026Architectural Wisdom: A Framework for Governing Optimization in AI Systems15 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026Epistemic Regret Minimization: Label-Free Causal Critique Beyond Outcome Reward
- 2026An Introduction to Causal Reinforcement Learning
- 2026When to Forget: A Memory Governance Primitive
- 2026Memory Reward Inflation in Self-Improving LLM Agents
- 2026Causal Retention in Interactive Agents: Interface Factorization and Selective Adaptation
- 2026The Epistemics of Agent Memory: Measuring, and Governing, the Consolidation Decision in Long-Horizon LLM Agents
- 2026Trajectory-Relative Hindsight Distillation for Agentic Reinforcement Learning
- 2024The Fallacy of Minimizing Cumulative Regret in the Sequential Task Setting