mlsanityData to 14 Oct 2026

AdaRefiner: Refining Decisions of Language Models with Adaptive Feedback

arXiv 2309.17176 ↗First posted 29 Sep 2023cs.AI cs.CLcited by 10

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated significant success across various domains. However, their application in complex decision-making tasks frequently necessitates intricate prompt engineering or fine-tuning, leading to challenges in unseen downstream tasks and heavy demands on computational resources. Meanwhile, Reinforcement Learning (RL) has been recognized as effective in decision-making problems but struggles in environments with sparse rewards, such as open-world games. To overcome these challenges, we introduce AdaRefiner, a novel framework designed to enhance the synergy between LLMs and RL feedback. The key component of AdaRefiner is a lightweight Adapter Language Model (LM), which automatically refines task comprehension based on feedback from RL agents. This method mitigates the need for intricate prompt engineering and intensive LLM fine-tuning while maintaining the LLMs' generalization abilities and enhancing their decision-making capabilities in downstream tasks. Empirical evaluations of AdaRefiner on 22 diverse tasks within the open-world game Crafter have demonstrated its superior effectiveness, especially in guiding agents towards higher-level and common-sense skills. Our work makes contributions to the automatic self-refinement of LLMs with RL feedback, offering a more adaptable and efficient solution for complex decision-making problems.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 32 of its 46 references that are in the map. First posted on arXiv 29 Sep 2023.

Builds on

32 of 46 references.

References · 32

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  6. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  8. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  9. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  10. 2023Mastering Diverse Domains through World Modelscited by 1,253
  11. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  12. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
20 more
  1. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
  2. 2018Exploration by Random Network Distillationcited by 993
  3. 2023BloombergGPT: A Large Language Model for Financecited by 939
  4. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  5. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 781
  6. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 398
  7. 2020Planning to Explore via Self-Supervised World Modelscited by 368
  8. 2023MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedbackcited by 297
  9. 2020Language Conditioned Imitation Learning over Unstructured Datacited by 209
  10. 2021Benchmarking the Spectrum of Agent Capabilitiescited by 200
  11. 2023AdaPlanner: Adaptive Planning from Feedback with Language Modelscited by 196
  12. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 166
  13. 2021Skill Induction and Planning with Latent Languagecited by 99
  14. 2023Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modellingcited by 88
  15. 2020Human Instruction-Following with Deep Reinforcement Learning via Transfer-Learning from Textcited by 59
  16. 2022LMPriors: Pre-Trained Language Models as Task-Specific Priorscited by 56
  17. 2023FlexKBQA: A Flexible LLM-Powered Framework for Few-Shot Knowledge Base Question Answeringcited by 51
  18. 2021Grounding Language to Entities and Dynamics for Generalization in Reinforcement Learningcited by 45
  19. 2019ACTRCE: Augmenting Experience via Teacher's Advice For Multi-Goal Reinforcement Learningcited by 17
  20. 2021SILG: The Multi-environment Symbolic Interactive Language Grounding Benchmarkcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 6; 2026: 1.

10 papers.

  1. 2024VideoGameBunny: Towards vision assistants for video gamescited by 11
  2. 2024A Large-Scale Simulation on Large Language Models for Decision-Making in Political Sciencecited by 10
  3. 2025The Evolving Landscape of LLM- and VLM-Integrated Reinforcement Learningcited by 10
  4. 2025CyberV: Cybernetics for Test-time Scaling in Video Understandingcited by 8
  5. 2024World Models with Hints of Large Language Models for Goal Achievingcited by 6
  6. 2025Causal-aware Large Language Models: Enhancing Decision-Making Through Learning, Adapting and Actingcited by 6
  7. 2025Expanding the Action Space of LLMs to Reason Beyond Languagecited by 1
  8. 2025Subgoal Graph-Augmented Planning for LLM-Guided Open-World Reinforcement Learningcited by 1
  9. 2026Retrieval-Augmented Interpretable Learning: Towards Task-Specific Zero-Shot Models in Healthcarecited by 1
  10. 2025Goal-Guided Efficient Exploration via Large Language Model in Reinforcement Learningcited by 0