# Intrinsic Credit Assignment for Long Horizon Interaction

arXiv: https://arxiv.org/abs/2602.12342 · HTML: https://mlsanity.com/paper/2602.12342/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Feb 2026.

## Built on by (most cited first, 9 of 9)

- [Aligning Language Models from User Interactions](https://mlsanity.com/paper/2603.12273/) (2026)
- [Agentic Forecasting using Sequential Bayesian Updating of Linguistic Beliefs](https://mlsanity.com/paper/2604.18576/) (2026)
- [Self-Induced Outcome Potential: Turn-Level Credit Assignment for Agents without Verifiers](https://mlsanity.com/paper/2605.04984/) (2026)
- [TAPO: Tool-Aware Policy Optimization via Credit Transfer for Multimodal Search Agents](https://mlsanity.com/paper/2606.05784/) (2026)
- [Amortising Bayesian Experimental Design for Sequential Information Gathering in LLMs](https://mlsanity.com/paper/2607.03426/) (2026)
- [QVal: Cheaply Evaluating Dense Supervision Signals for Long-Horizon LLM Agents](https://mlsanity.com/paper/2606.32034/) (2026)
- [Counterfactual Shapley Credit Assignment](https://mlsanity.com/paper/2607.16999/) (2026)
- [Counterfactual Rollout Replay: Forkable Environments as Free Process Rewards for Software Engineering Agents](https://mlsanity.com/paper/2609.33875/) (2026)
- [Dr.Credit: Rubric-Grounded Process Credit Assignment for Deep Research Agents](https://mlsanity.com/paper/2609.34296/) (2026)

## Abstract (arXiv)

How can we train agents to navigate uncertainty over long horizons? In this work, we propose DeltaBelief-RL, which leverages a language model's own intrinsic beliefs to reward intermediate progress. Our method utilizes the change in the probability an agent assigns to the target solution for credit assignment. By training on synthetic interaction data, DeltaBelief-RL teaches information-seeking capabilities that consistently outperform purely outcome-based rewards for Reinforcement Learning, with improvements generalizing to out-of-distribution applications ranging from customer service to personalization. Notably, the performance continues to improve as we scale test-time interactions beyond the training horizon, with interaction-efficiency increasing even on Pass@k metrics. Overall, our work introduces a scalable training strategy for navigating uncertainty over a long-horizon, by enabling credit assignment to intermediate actions via intrinsic DeltaBelief rewards.