mlsanityData to 14 Oct 2026

Policy Learning with a Language Bottleneck

arXiv 2405.04118 ↗First posted 7 May 2024cs.LG cs.AI cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

Modern AI systems such as self-driving cars and game-playing agents can achieve superhuman performance, but often lack human-like generalization, interpretability, and inter-operability with human users. Inspired by the rich interactions between language and decision-making in humans, we introduce Policy Learning with a Language Bottleneck (PLLB), a framework enabling AI agents to generate linguistic rules that capture the high-level strategies underlying rewarding behaviors. PLLB alternates between a *rule generation* step guided by language models, and an *update* step where agents learn new policies guided by rules, even when a rule is insufficient to describe an entire complex policy. Across five diverse tasks, including a two-player signaling game, maze navigation, image reconstruction, and robot grasp planning, we show that PLLB agents are not only able to learn more interpretable and generalizable behaviors, but can also share the learned rules with human users, enabling more effective human-AI coordination. We provide source code for our experiments at https://github.com/meghabyte/bottleneck .

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 37 of its 72 references that are in the map. First posted on arXiv 7 May 2024.

Builds on

37 of 72 references.

References · 37

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  4. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  6. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  7. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  8. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  9. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  10. 2018BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learningcited by 244
  11. 2023Physically Grounded Vision-Language Models for Robotic Manipulationcited by 165
  12. 2019A Survey of Reinforcement Learning Informed by Natural Languagecited by 154
25 more
  1. 2019Language as an Abstraction for Hierarchical Deep Reinforcement Learningcited by 138
  2. 2023Chain of Code: Reasoning with a Language Model-Augmented Code Emulatorcited by 123
  3. 2017Grasp Pose Detection in Point Cloudscited by 111
  4. 2023Motif: Intrinsic Motivation from Artificial Intelligence Feedbackcited by 102
  5. 2017Grounded Language Learning in a Simulated 3D Worldcited by 100
  6. 2021Skill Induction and Planning with Latent Languagecited by 99
  7. 2023Eliciting Human Preferences with Language Modelscited by 92
  8. 2020Aligning Superhuman AI with Human Behavior: Chess as a Model Systemcited by 79
  9. 2020Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Explorationcited by 68
  10. 2023Language Instructed Reinforcement Learning for Human-AI Coordinationcited by 63
  11. 2023Goal Driven Discovery of Distributional Differences via Language Descriptionscited by 62
  12. 2023Describing Differences in Image Sets with Natural Languagecited by 53
  13. 2022Semantic Exploration from Language Abstractions and Pretrained Representationscited by 51
  14. 2021Leveraging Language to Learn Program Abstractions and Search Heuristicscited by 45
  15. 2023Large Language Models Are Semi-Parametric Reinforcement Learning Agentscited by 42
  16. 2021Tell me why! Explanations support learning relational and causal structurecited by 36
  17. 2020Ask Your Humans: Using Human Instructions to Improve Generalization in Reinforcement Learningcited by 32
  18. 2023Thought Cloning: Learning to Think while Acting by Imitating Human Thinkingcited by 30
  19. 2019Continual adaptation for efficient machine communicationcited by 28
  20. 2020Pragmatic Issue-Sensitive Image Captioningcited by 15
  21. 2022Language and Culture Internalisation for Human-Like Autotelic AIcited by 14
  22. 2022Human-AI Coordination via Human-Regularized Search and Learningcited by 8
  23. 2021Learning to solve complex tasks by growing knowledge culturally across generationscited by 6
  24. 2022Explainability Via Causal Self-Talkcited by 6
  25. 2022Teachable Reinforcement Learning via Advice Distillationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 0; 2026: 1.

4 papers.

  1. 2024FlowRetrieval: Flow-Guided Data Retrieval for Few-Shot Imitation Learningcited by 33
  2. 2024Efficacy of Language Model Self-Play in Non-Zero-Sum Gamescited by 15
  3. 2024RuAG: Learned-rule-augmented Generation for Large Language Modelscited by 5
  4. 2026Inner Speech as Behavior Guides: Steerable Imitation of Diverse Behaviors for Human-AI coordinationcited by 2