# Retrieval Augmented Learning: A Retrial-based Large Language Model Self-Supervised Learning and Autonomous Knowledge Generation

arXiv: https://arxiv.org/abs/2505.01073 · HTML: https://mlsanity.com/paper/2505.01073/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation.
- Builds on 22 of its 32 references that are in the map.
- First posted on arXiv 2 May 2025.

## Placed in

- Retrieval-augmented generation / [RAG pipeline design and evaluation](https://mlsanity.com/rag/rag-pipeline-design-and-evaluation/)

## Builds on (most cited first, 22 of 22)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [ChatDev: Communicative Agents for Software Development](https://mlsanity.com/paper/2307.07924/) (2023)
- [Value-Decomposition Networks For Cooperative Multi-Agent Learning](https://mlsanity.com/paper/1706.05296/) (2017)
- [Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory](https://mlsanity.com/paper/2305.17144/) (2023)
- [TTRL: Test-Time Reinforcement Learning](https://mlsanity.com/paper/2504.16084/) (2025)
- [Exploring Large Language Models for Communication Games: An Empirical Study on Werewolf](https://mlsanity.com/paper/2309.04658/) (2023)
- [TradingAgents: Multi-Agents LLM Financial Trading Framework](https://mlsanity.com/paper/2412.20138/) (2024)
- [Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling](https://mlsanity.com/paper/2502.06703/) (2025)
- [Reinforcement Learning Enhanced LLMs: A Survey](https://mlsanity.com/paper/2412.10400/) (2024)
- [How Much Can RAG Help the Reasoning of LLM?](https://mlsanity.com/paper/2410.02338/) (2024)
- [LLaMA Rider: Spurring Large Language Models to Explore the Open World](https://mlsanity.com/paper/2310.08922/) (2023)
- [SwarmBrain: Embodied agent for real-time strategy game StarCraft II via large language models](https://mlsanity.com/paper/2401.17749/) (2024)
- [LLM-PySC2: Starcraft II learning environment for Large Language Models](https://mlsanity.com/paper/2411.05348/) (2024)
- [SMAC-R1: The Emergence of Intelligence in Decision-Making Tasks](https://mlsanity.com/paper/2410.16024/) (2024)
- [SMAC-Hard: Enabling Mixed Opponent Strategy Script and Self-play on SMAC](https://mlsanity.com/paper/2412.17707/) (2024)
- [Parallelized Planning-Acting for Efficient LLM-based Multi-Agent Systems in Minecraft](https://mlsanity.com/paper/2503.03505/) (2025)
- [Hierarchical Expert Prompt for Large-Language-Model: An Approach Defeat Elite AI in TextStarCraft II for the First Time](https://mlsanity.com/paper/2502.11122/) (2025)
- [Reflection of Episodes: Learning to Play Game from Expert and Self Experiences](https://mlsanity.com/paper/2502.13388/) (2025)

## Built on by (most cited first, 1 of 1)

- [RASPRef: Retrieval-Augmented Self-Supervised Prompt Refinement for Large Reasoning Models](https://mlsanity.com/paper/2603.27008/) (2026)

## Abstract (arXiv)

The lack of domain-specific data in the pre-training of Large Language Models (LLMs) severely limits LLM-based decision systems in specialized applications, while post-training a model in the scenarios requires significant computational resources. In this paper, we present Retrial-Augmented Learning (RAL), a reward-free self-supervised learning framework for LLMs that operates without model training. By developing Retrieval-Augmented Generation (RAG) into a module for organizing intermediate data, we realized a three-stage autonomous knowledge generation of proposing a hypothesis, validating the hypothesis, and generating the knowledge. The method is evaluated in the LLM-PySC2 environment, a representative decision-making platform that combines sufficient complexity with domain-specific knowledge requirements. Experiments demonstrate that the proposed method effectively reduces hallucination by generating and utilizing validated knowledge, and increases decision-making performance at an extremely low cost. Meanwhile, the approach exhibits potential in out-of-distribution(OOD) tasks, robustness, and transferability, making it a cost-friendly but effective solution for decision-making problems and autonomous knowledge generation.