# LLaMA Rider: Spurring Large Language Models to Explore the Open World

arXiv: https://arxiv.org/abs/2310.08922 · HTML: https://mlsanity.com/paper/2310.08922/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 33 references that are in the map.
- First posted on arXiv 13 Oct 2023.

## Builds on (most cited first, 30 of 31)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents](https://mlsanity.com/paper/2207.01206/) (2022)
- [Mind2Web: Towards a Generalist Agent for the Web](https://mlsanity.com/paper/2306.06070/) (2023)
- [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022)
- [Inner Monologue: Embodied Reasoning through Planning with Language Models](https://mlsanity.com/paper/2207.05608/) (2022)
- [Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents](https://mlsanity.com/paper/2201.07207/) (2022)
- [ProgPrompt: Generating Situated Robot Task Plans using Large Language Models](https://mlsanity.com/paper/2209.11302/) (2022)
- [MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge](https://mlsanity.com/paper/2206.08853/) (2022)
- [VirtualHome: Simulating Household Activities via Programs](https://mlsanity.com/paper/1806.07011/) (2018)
- [Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents](https://mlsanity.com/paper/2302.01560/) (2023)
- [Language Models can Solve Computer Tasks](https://mlsanity.com/paper/2303.17491/) (2023)
- [Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory](https://mlsanity.com/paper/2305.17144/) (2023)
- [BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning](https://mlsanity.com/paper/1810.08272/) (2018)
- [Pre-Trained Language Models for Interactive Decision-Making](https://mlsanity.com/paper/2202.01771/) (2022)
- [MineRL: A Large-Scale Dataset of Minecraft Demonstrations](https://mlsanity.com/paper/1907.13440/) (2019)
- [Grounding Large Language Models in Interactive Environments with Online Reinforcement Learning](https://mlsanity.com/paper/2302.02662/) (2023)
- [Language Models Meet World Models: Embodied Experiences Enhance Language Models](https://mlsanity.com/paper/2305.10626/) (2023)
- [Mind's Eye: Grounded Language Model Reasoning through Simulation](https://mlsanity.com/paper/2210.05359/) (2022)
- [Collaborating with language models for embodied reasoning](https://mlsanity.com/paper/2302.00763/) (2023)

## Built on by (most cited first, 21 of 21)

- [A Survey on Large Language Model-Based Game Agents](https://mlsanity.com/paper/2404.02039/) (2024)
- [RL-GPT: Integrating Reinforcement Learning and Code-as-policy](https://mlsanity.com/paper/2402.19299/) (2024)
- [LLM-Empowered State Representation for Reinforcement Learning](https://mlsanity.com/paper/2407.13237/) (2024)
- [Odyssey: Empowering Minecraft Agents with Open-World Skills](https://mlsanity.com/paper/2407.15325/) (2024)
- [BattleAgentBench: A Benchmark for Evaluating Cooperation and Competition Capabilities of Language Models in Multi-Agent Systems](https://mlsanity.com/paper/2408.15971/) (2024)
- [A Survey on Game Playing Agents and Large Models: Methods, Applications, and Challenges](https://mlsanity.com/paper/2403.10249/) (2024)
- [UniCode: Learning a Unified Codebook for Multimodal Large Language Models](https://mlsanity.com/paper/2403.09072/) (2024)
- [VistaWise: Building Cost-Effective Agent with Cross-Modal Knowledge Graph for Minecraft](https://mlsanity.com/paper/2508.18722/) (2025)
- [LARM: Large Auto-Regressive Model for Long-Horizon Embodied Intelligence](https://mlsanity.com/paper/2405.17424/) (2024)
- [Can foundation models actively gather information in interactive environments to test hypotheses?](https://mlsanity.com/paper/2412.06438/) (2024)
- [ProRefine: Inference-Time Prompt Refinement with Textual Feedback](https://mlsanity.com/paper/2506.05305/) (2025)
- [LLM-Driven Stationarity-Aware Expert Demonstrations for Multi-Agent Reinforcement Learning in Mobile Systems](https://mlsanity.com/paper/2511.19368/) (2025)
- [Large Language Models Think Too Fast To Explore Effectively](https://mlsanity.com/paper/2501.18009/) (2025)
- [Taking Notes Brings Focus? Towards Multi-Turn Multimodal Dialogue Learning](https://mlsanity.com/paper/2503.07002/) (2025)
- [Bottom-Up Synthesis of Knowledge-Grounded Task-Oriented Dialogues with Iteratively Self-Refined Prompts](https://mlsanity.com/paper/2504.14375/) (2025)
- [Experience Transfer for Multimodal LLM Agents in Minecraft Game](https://mlsanity.com/paper/2604.05533/) (2026)
- [QuadrupedGPT: Towards a Versatile Quadruped Agent in Open-ended Worlds](https://mlsanity.com/paper/2406.16578/) (2024)
- [VLM Q-Learning: Aligning Vision-Language Models for Interactive Decision-Making](https://mlsanity.com/paper/2505.03181/) (2025)
- [Towards Generalist Game Players: An Investigation of Foundation Models in the Game Multiverse](https://mlsanity.com/paper/2605.09965/) (2026)
- [Retrieval Augmented Learning: A Retrial-based Large Language Model Self-Supervised Learning and Autonomous Knowledge Generation](https://mlsanity.com/paper/2505.01073/) (2025)
- [LLM-Cave: A benchmark and light environment for large language models reasoning and decision-making system](https://mlsanity.com/paper/2511.22598/) (2025)

## Abstract (arXiv)

Recently, various studies have leveraged Large Language Models (LLMs) to help decision-making and planning in environments, and try to align the LLMs' knowledge with the world conditions. Nonetheless, the capacity of LLMs to continuously acquire environmental knowledge and adapt in an open world remains uncertain. In this paper, we propose an approach to spur LLMs to explore the open world, gather experiences, and learn to improve their task-solving capabilities. In this approach, a multi-round feedback-revision mechanism is utilized to encourage LLMs to actively select appropriate revision actions guided by feedback information from the environment. This facilitates exploration and enhances the model's performance. Besides, we integrate sub-task relabeling to assist LLMs in maintaining consistency in sub-task planning and help the model learn the combinatorial nature between tasks, enabling it to complete a wider range of tasks through training based on the acquired exploration experiences. By evaluation in Minecraft, an open-ended sandbox world, we demonstrate that our approach LLaMA-Rider enhances the efficiency of the LLM in exploring the environment, and effectively improves the LLM's ability to accomplish more tasks through fine-tuning with merely 1.3k instances of collected data, showing minimal training costs compared to the baseline using reinforcement learning.