# TRAD: Enhancing LLM Agents with Step-Wise Thought Retrieval and Aligned Decision

arXiv: https://arxiv.org/abs/2403.06221 · HTML: https://mlsanity.com/paper/2403.06221/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 12 of its 48 references that are in the map.
- First posted on arXiv 10 Mar 2024.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 12 of 12)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [Graph of Thoughts: Solving Elaborate Problems with Large Language Models](https://mlsanity.com/paper/2308.09687/) (2023)
- [LLM+P: Empowering Large Language Models with Optimal Planning Proficiency](https://mlsanity.com/paper/2304.11477/) (2023)
- [Large Language Models for Information Retrieval: A Survey](https://mlsanity.com/paper/2308.07107/) (2023)
- [Everything of Thoughts: Defying the Law of Penrose Triangle for Thought Generation](https://mlsanity.com/paper/2311.04254/) (2023)

## Built on by (most cited first, 23 of 23)

- [MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memory](https://mlsanity.com/paper/2601.03192/) (2026)
- [Agent KB: Leveraging Cross-Domain Experience for Agentic Problem Solving](https://mlsanity.com/paper/2507.06229/) (2025)
- [AgentNet: Decentralized Evolutionary Coordination for LLM-based Multi-Agent Systems](https://mlsanity.com/paper/2504.00587/) (2025)
- [A Survey of AI Agent Protocols](https://mlsanity.com/paper/2504.16736/) (2025)
- [RuleArena: A Benchmark for Rule-Guided Reasoning with LLMs in Real-World Scenarios](https://mlsanity.com/paper/2412.08972/) (2024)
- [Agentic Information Retrieval](https://mlsanity.com/paper/2410.09713/) (2024)
- [Self-Generated In-Context Examples Improve LLM Agents for Sequential Decision-Making Tasks](https://mlsanity.com/paper/2505.00234/) (2025)
- [Agent Exchange: Shaping the Future of AI Agent Economics](https://mlsanity.com/paper/2507.03904/) (2025)
- [Self-Evolving World Models for LLM Agent Planning](https://mlsanity.com/paper/2606.30639/) (2026)
- [Structurally Aligned Subtask-Level Memory for Software Engineering Agents](https://mlsanity.com/paper/2602.21611/) (2026)
- [HiPlan: Hierarchical Planning for LLM-Based Agents with Adaptive Global-Local Guidance](https://mlsanity.com/paper/2508.19076/) (2025)
- [Boosting Virtual Agent Learning and Reasoning: A Step-Wise, Multi-Dimensional, and Generalist Reward Model with Benchmark](https://mlsanity.com/paper/2503.18665/) (2025)
- [Mining Intrinsic Rewards from LLM Hidden States for Efficient Best-of-N Sampling](https://mlsanity.com/paper/2505.12225/) (2025)
- [World-Model-Augmented Web Agents with Action Correction](https://mlsanity.com/paper/2602.15384/) (2026)
- [Epistemic Context Learning: Building Trust the Right Way in LLM-Based Multi-Agent Systems](https://mlsanity.com/paper/2601.21742/) (2026)
- [ActMem: Bridging the Gap Between Memory Retrieval and Reasoning in LLM Agents](https://mlsanity.com/paper/2603.00026/) (2026)
- [Bridging the Agent-World Gap: Text World Models for LLM-based Agents](https://mlsanity.com/paper/2606.09032/) (2026)
- [Leveraging In-Context Learning for Language Model Agents](https://mlsanity.com/paper/2506.13109/) (2025)
- [Inference-Time Distillation: Cost-Efficient Agents Without Fine-Tuning or Manual Prompt Engineering](https://mlsanity.com/paper/2512.02543/) (2025)
- [LLM-R: A Framework for Domain-Adaptive Maintenance Scheme Generation Combining Hierarchical Agents and RAG](https://mlsanity.com/paper/2411.04476/) (2024)
- [Reflecting with Two Voices: A Co-Adaptive Dual-Strategy Framework for LLM-Based Agent Decision Making](https://mlsanity.com/paper/2512.08366/) (2025)
- [Foundation-Model-Based Agents in Industrial Automation: Purposes, Capabilities, and Open Challenges](https://mlsanity.com/paper/2605.02592/) (2026)
- [Leveraging Trajectory Graphs for Pre-Execution Error Diagnosis in Agentic LLM Systems](https://mlsanity.com/paper/2607.27443/) (2026)

## Abstract (arXiv)

Numerous large language model (LLM) agents have been built for different tasks like web navigation and online shopping due to LLM's wide knowledge and text-understanding ability. Among these works, many of them utilize in-context examples to achieve generalization without the need for fine-tuning, while few of them have considered the problem of how to select and effectively utilize these examples. Recently, methods based on trajectory-level retrieval with task meta-data and using trajectories as in-context examples have been proposed to improve the agent's overall performance in some sequential decision making tasks. However, these methods can be problematic due to plausible examples retrieved without task-specific state transition dynamics and long input with plenty of irrelevant context. In this paper, we propose a novel framework (TRAD) to address these issues. TRAD first conducts Thought Retrieval, achieving step-level demonstration selection via thought matching, leading to more helpful demonstrations and less irrelevant input noise. Then, TRAD introduces Aligned Decision, complementing retrieved demonstration steps with their previous or subsequent steps, which enables tolerance for imperfect thought and provides a choice for balance between more context and less noise. Extensive experiments on ALFWorld and Mind2Web benchmarks show that TRAD not only outperforms state-of-the-art models but also effectively helps in reducing noise and promoting generalization. Furthermore, TRAD has been deployed in real-world scenarios of a global business insurance company and improves the success rate of robotic process automation.