# When History Fails to Become Experience: Action Calibration in Language Agents

arXiv: https://arxiv.org/abs/2610.02769 · HTML: https://mlsanity.com/paper/2610.02769/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 18 references that are in the map.
- First posted on arXiv 2 Oct 2026.

## Builds on (most cited first, 16 of 16)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents](https://mlsanity.com/paper/2207.01206/) (2022)
- [ALFWorld: Aligning Text and Embodied Environments for Interactive Learning](https://mlsanity.com/paper/2010.03768/) (2020)
- [Group-in-Group Policy Optimization for LLM Agent Training](https://mlsanity.com/paper/2505.10978/) (2025)
- [AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents](https://mlsanity.com/paper/2407.18901/) (2024)
- [ScienceWorld: Is your Agent Smarter than a 5th Grader?](https://mlsanity.com/paper/2203.07540/) (2022)
- [MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents](https://mlsanity.com/paper/2602.02474/) (2026)
- [MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memory](https://mlsanity.com/paper/2601.03192/) (2026)
- [Breaking Contextual Inertia: Reinforcement Learning with Single-Turn Anchors for Stable Multi-Turn Interaction](https://mlsanity.com/paper/2603.04783/) (2026)
- [Mitigating Conversational Inertia in Multi-Turn Agents](https://mlsanity.com/paper/2602.03664/) (2026)
- [HyperSkill: Self-Evolving LLM Agents via Hypergraph-Structured Skill Memory](https://mlsanity.com/paper/2608.16114/) (2026)
- [Feedback That Backfires: Why Small Language Model Agents Repeat the Call They Just Watched Fail](https://mlsanity.com/paper/2608.23651/) (2026)
- [Positive Experience Reflection for Agents in Interactive Text Environments](https://mlsanity.com/paper/2411.02223/) (2024)
- [APEx: Distillation of Agent Procedural Experience for Adaptive Deep Research Question Answering](https://mlsanity.com/paper/2609.02253/) (2026)
- [Seeing Isn't Believing: Mitigating Belief Inertia via Active Intervention in Embodied Agents](https://mlsanity.com/paper/2604.17252/) (2026)

## Abstract (arXiv)

Language agents should draw on prior attempts and environmental feedback to improve subsequent decisions within the same task. However, providing additional interaction history can sometimes reduce task success, suggesting that agents do not consistently use this information effectively. To investigate this limitation, we examine how agents use history. We find that history improves task completion overall, yet much of this benefit persists even when past actions are shuffled. Disrupting the correspondence between actions and observations causes only a modest decline in task success. We therefore hypothesize that agents do not reliably connect past actions with their outcomes when deciding how to proceed. To test this hypothesis, we explicitly label each returned observation as the outcome of the preceding action. This simple annotation improves task success and reduces next-action repetition without introducing new environmental information. Building on this insight, we introduce a learned calibrator that explicitly reassesses past actions and selectively records experience to guide subsequent decisions, improving task success beyond outcome labeling alone.