# Credit Without Ground Truth: Auditing Step-Level Credit Assignment in LLM Agents Against Executed Replay

arXiv: https://arxiv.org/abs/2608.19760 · HTML: https://mlsanity.com/paper/2608.19760/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 10 of its 46 references that are in the map.
- First posted on arXiv 20 Aug 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 10 of 10)

- [Hindsight Credit Assignment for Long-Horizon LLM Agents](https://mlsanity.com/paper/2603.08754/) (2026)
- [From Reasoning to Agentic: Credit Assignment in Reinforcement Learning for Large Language Models](https://mlsanity.com/paper/2604.09459/) (2026)
- [Causal Agent Replay: Counterfactual Attribution for LLM-Agent Failures](https://mlsanity.com/paper/2606.08275/) (2026)
- [Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?](https://mlsanity.com/paper/2607.09996/) (2026)
- [StepOPSD: Step-Aware Online Preference Self-Distillation for Agent Reinforcement Learning](https://mlsanity.com/paper/2605.27140/) (2026)
- [Verified Critical Step Optimization for LLM Agents](https://mlsanity.com/paper/2602.03412/) (2026)
- [Process-Supervised Reinforcement Learning for Interactive Multimodal Tool-Use Agents](https://mlsanity.com/paper/2509.14480/) (2025)
- [Counterfactual Credit Policy Optimization for Multi-Agent Collaboration](https://mlsanity.com/paper/2603.21563/) (2026)
- [Exact Is Easier: Credit Assignment for Cooperative LLM Agents](https://mlsanity.com/paper/2603.06859/) (2026)
- [CARL: Criticality-Aware Agentic Reinforcement Learning](https://mlsanity.com/paper/2512.04949/) (2025)

## Built on by (most cited first, 7 of 7)

- [S3Gym: Can LLMs Turn Self-Testing and Self-Judging into Self-Improvement?](https://mlsanity.com/paper/2608.31100/) (2026)
- [Why Didn't It Check? Unsupported Final Claims and Their Repair in Two Tool-Equipped Language Models](https://mlsanity.com/paper/2608.27768/) (2026)
- [Critical-State RL: Diagnosing Trainable States for Multi-Turn Tool Use](https://mlsanity.com/paper/2609.24985/) (2026)
- [Auditing Proxy-Based Validation Across Text Spans](https://mlsanity.com/paper/2609.25808/) (2026)
- [R2 Flow: Recursive Self-Improvement via Recursive Skill Evolution](https://mlsanity.com/paper/2609.33867/) (2026)
- [ReplayLens: Auditing Agents' Use of Outcomes](https://mlsanity.com/paper/2609.34177/) (2026)
- [GraphOPD: Graph-Augmented On-Policy Distillation for LLM Agents](https://mlsanity.com/paper/2610.08959/) (2026)

## Abstract (arXiv)

Audited against policy-conditional ground truth from executed replay in a single-agent tool environment (ALFWorld), none of the step-level credit signals we audit -- LLM-judge scores, outcome-conditioned logprob ratios, or the policy's own confidence -- shows reliable incremental fidelity beyond its own marginal-matched shuffled control. Correcting for replay-target reliability leaves implicit fidelity bounded near zero and judge fidelity inconclusive at the achieved target reliability. Existing evaluations grade these signals against annotated step *correctness*; we audit them against step *contribution* -- what re-sampling the policy's own alternatives at each decision point and rolling forward changes about the outcome -- and they come apart. The ground truth is structured: 30.5% of decision points where it is defined exhibit a nonzero replay contrast at the achieved sampling resolution, and measurability is model-dependent -- the fraction of points with no policy-supported counterfactual differs twofold (13.1% vs. 26.8%) between two similar-scale policies. The failure mode is identifiable: implicit credit echoes the policy's fluency (median rank correlation +0.75, replicating at +0.70 in a second family under a corrected instrument), while outcome conditioning adds no causal information (partial correlation -0.004, Qwen). A confidence-only router recovers pivotal steps at chance level, but cuts judge cost by 13.1% per turn (14.0% per trajectory). In a seven-arm pre-registered training experiment, no arm reliably outperforms the untrained policy, and the checkpoints' apparent instrument signature is statistically consistent with mediation by effective training dose in this design -- sparser credit retains fewer examples, an order-of-magnitude spread in optimizer steps -- not credit content. Comparisons of credit rules must match effective sample size, or they measure dose, not credit.