mlsanityData to 14 Oct 2026

LLF-Bench: Benchmark for Interactive Learning from Language Feedback

arXiv 2312.06853 ↗First posted 11 Dec 2023cs.AIcited by 33

Not on the map. Why?

Abstract (arXiv)

We introduce a new benchmark, LLF-Bench (Learning from Language Feedback Benchmark; pronounced as "elf-bench"), to evaluate the ability of AI agents to interactively learn from natural language feedback and instructions. Learning from language feedback (LLF) is essential for people, largely because the rich information this feedback provides can help a learner avoid much of trial and error and thereby speed up the learning process. Large Language Models (LLMs) have recently enabled AI agents to comprehend natural language -- and hence AI agents can potentially benefit from language feedback during learning like humans do. But existing interactive benchmarks do not assess this crucial capability: they either use numeric reward feedback or require no learning at all (only planning or information retrieval). LLF-Bench is designed to fill this omission. LLF-Bench is a diverse collection of sequential decision-making tasks that includes user recommendation, poem writing, navigation, and robot control. The objective of an agent is to interactively solve these tasks based on their natural-language instructions and the feedback received after taking actions. Crucially, to ensure that the agent actually "learns" from the feedback, LLF-Bench implements several randomization techniques (such as paraphrasing and environment randomization) to ensure that the task isn't familiar to the agent and that the agent is robust to various verbalizations. In addition, LLF-Bench provides a unified OpenAI Gym interface for all its tasks and allows the users to easily configure the information the feedback conveys (among suggestion, explanation, and instantaneous performance) to study how agents respond to different types of feedback. Together, these features make LLF-Bench a unique research platform for developing and testing LLF agents.

Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). Builds on 39 of its 54 references that are in the map. First posted on arXiv 11 Dec 2023.

Builds on

39 of 54 references.

References · 39

  1. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  2. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  3. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  4. 2016Concrete Problems in AI Safetycited by 1,642
  5. 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
  6. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
  7. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
  8. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  9. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
  10. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
  11. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 498
  12. 2018TextWorld: A Learning Environment for Text-based Gamescited by 310
27 more
  1. 2023MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedbackcited by 297
  2. 2018Reinforcement Learning on Web Interfaces Using Workflow-Guided Explorationcited by 284
  3. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 239
  4. 2020The NetHack Learning Environmentcited by 198
  5. 2023OpenAGI: When LLM Meets Domain Expertscited by 197
  6. 2023Language-Driven Representation Learning for Roboticscited by 165
  7. 2022Ask Me Anything: A simple strategy for prompting language modelscited by 155
  8. 2023Fundamental Limitations of Alignment in Large Language Modelscited by 155
  9. 2018Mapping Instructions to Actions in 3D Environments with Visual Goal Predictioncited by 101
  10. 2023LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Modelscited by 88
  11. 2018Batch Active Preference-Based Learning of Reward Functionscited by 87
  12. 2017Mapping Instructions and Visual Observations to Actions with Reinforcement Learningcited by 81
  13. 2018Learning to Understand Goal Specifications by Modelling Rewardcited by 80
  14. 2017Gated-Attention Architectures for Task-Oriented Language Groundingcited by 72
  15. 2015Language Understanding for Text-based Games Using Deep Reinforcement Learningcited by 71
  16. 2021Feedback in Imitation Learning: The Three Regimes of Covariate Shiftcited by 68
  17. 2023"No, to the Right" -- Online Language Corrections for Robotic Manipulation via Shared Autonomycited by 60
  18. 2021Grounding Language to Entities and Dynamics for Generalization in Reinforcement Learningcited by 45
  19. 2023The language of prompting: What linguistic properties make a prompt successful?cited by 44
  20. 2016Learning Language Games through Interactioncited by 43
  21. 2017From Language to Programs: Bridging Reinforcement Learning and Maximum Marginal Likelihoodcited by 41
  22. 2020Learning Rewards from Linguistic Feedbackcited by 41
  23. 2023Interactive Robot Learning from Verbal Correctioncited by 29
  24. 2021LILA: Language-Informed Latent Actionscited by 27
  25. 2021Interactive Learning from Activity Descriptioncited by 26
  26. 2017Representation Learning for Grounded Spatial Reasoningcited by 23
  27. 2020Safe Reinforcement Learning with Natural Language Constraintscited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 14; 2025: 8; 2026: 11.

33 papers.

Sort:

By area:

  1. 2024AI Agents That Mattercited by 219
  2. 2025Survey on Evaluation of LLM-based Agentscited by 204
  3. 2025SWEET-RL: Training Multi-Turn LLM Agents on Collaborative Reasoning Taskscited by 118
  4. 2024Aligning LLM Agents by Learning Latent Preference from User Editscited by 69
  5. 2024Trace is the Next AutoDiff: Generative Optimization with Rich Feedback, Execution Traces, and LLMscited by 69
  6. 2024VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs of Thoughtcited by 55
  7. 2024On Overcoming Miscalibrated Conversational Priors in LLM-based Chatbotscited by 24
  8. 2024Beyond Prompts: Dynamic Conversational Benchmarking of Large Language Modelscited by 20
  9. 2025StoryBench: A Dynamic Benchmark for Evaluating Long-Term Memory with Multi Turnscited by 20
  10. 2024Tur[k]ingBench: A Challenge Benchmark for Web Agentscited by 18
  11. 2024Natural Language Reinforcement Learningcited by 17
  12. 2025Designing Domain-Specific Agents via Hierarchical Task Abstraction Mechanismcited by 16
21 more
  1. 2024Reflect-RL: Two-Player Online RL Fine-Tuning for LMscited by 13
  2. 2025Formalizing Learning from Language Feedback with Provable Guaranteescited by 13
  3. 2025Evolutionary Perspectives on the Evaluation of LLM-Based AI Agents: A Comprehensive Surveycited by 13
  4. 2024Teaching Embodied Reinforcement Learning Agents: Informativeness and Diversity of Language Usecited by 7
  5. 2024FIRE: A Dataset for Feedback Integration and Refinement Evaluation of Multimodal Modelscited by 6
  6. 2024Natural Language Reinforcement Learningcited by 5
  7. 2024Learning Goal-Conditioned Representations for Language Reward Modelscited by 2
  8. 2024A Survey on Complex Tasks for Goal-Directed Interactive Agentscited by 2
  9. 2025Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agentscited by 2
  10. 2026Learning Stateful Predictive Knowledge From Experiencecited by 2
  11. 2026Overcoming the Weakest-Link Effect in LLM-Driven Program Optimization via Heterogeneous Edit Recombinationcited by 2
  12. 2025A Signal Contract for Online Language Grounding and Discovery in Decision-Makingcited by 0
  13. 2026Failing to Explore: Language Models on Interactive Taskscited by 0
  14. 2026Language-Critique Imitation Learning from Suboptimal Demonstrationscited by 0
  15. 2026From Static Policies to Adaptive Priors in Offline Reinforcement Learningcited by 0
  16. 2026Benchmarking Prompt Optimization of Large Language Models With Chesscited by 0
  17. 2026JevSpawn: Adaptive Agentic Inference through Compositional Action Spacescited by 0
  18. 2026SPHERE: Adaptive VR Indoor Scene Generation via LLM-Enhanced Spatial Preference Learning and Human-in-the-Loop RLcited by 0
  19. 2026Learning What to Investigate Next: Meta-Reasoning for Long-Horizon Research Agentscited by 0
  20. 2026ServeLearnBench: How Well Can Agents Self-Improve from Serving Experience?cited by 0
  21. 2026Constraint Tree Exploration for Learning from Language Feedbackcited by 0