mlsanityData to 14 Oct 2026

lilGym: Natural Language Visual Reasoning with Reinforcement Learning

arXiv 2211.01994 ↗First posted 3 Nov 2022cs.LG cs.AI cs.CLcited by 3

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

We present lilGym, a new benchmark for language-conditioned reinforcement learning in visual environments. lilGym is based on 2,661 highly-compositional human-written natural language statements grounded in an interactive visual environment. We introduce a new approach for exact reward computation in every possible world state by annotating all statements with executable Python programs. Each statement is paired with multiple start states and reward functions to form thousands of distinct Markov Decision Processes of varying difficulty. We experiment with lilGym with different models and learning regimes. Our results and analysis show that while existing methods are able to achieve non-trivial performance, lilGym forms a challenging open problem. lilGym is available at https://lil.nlp.cornell.edu/lilgym/.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 38 of its 61 references that are in the map. First posted on arXiv 3 Nov 2022.

Builds on

38 of 61 references.

Closest ancestors · 1

Same direction, newest first

  1. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 37

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  5. 2015VQA: Visual Question Answeringcited by 2,506
  6. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  7. 2016OpenAI Gymcited by 1,736
  8. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  9. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
  10. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
  11. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  12. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
25 more
  1. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 649
  2. 2018A Corpus for Reasoning About Natural Language Grounded in Photographscited by 459
  3. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 414
  4. 2018TextWorld: A Learning Environment for Text-based Gamescited by 310
  5. 2018BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learningcited by 244
  6. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 239
  7. 2015Contextual Markov Decision Processescited by 220
  8. 2018Compositional Attention Networks for Machine Reasoningcited by 183
  9. 2019Interactive Fiction Games: A Colossal Adventurecited by 168
  10. 2017Inferring and Executing Programs for Visual Reasoningcited by 130
  11. 2019Stay on the Path: Instruction Fidelity in Vision-and-Language Navigationcited by 130
  12. 2019Learning to Speak and Act in a Fantasy Text Adventure Gamecited by 122
  13. 2017Grounded Language Learning in a Simulated 3D Worldcited by 100
  14. 2017Mapping Instructions and Visual Observations to Actions with Reinforcement Learningcited by 81
  15. 2015Language Understanding for Text-based Games Using Deep Reinforcement Learningcited by 71
  16. 2021Grounding Language to Entities and Dynamics for Generalization in Reinforcement Learningcited by 45
  17. 2021Leveraging Language to Learn Program Abstractions and Search Heuristicscited by 45
  18. 2018Guiding Policies with Language via Meta-Learningcited by 35
  19. 2018Learning to Map Context-Dependent Sentences to Executable Formal Queriescited by 23
  20. 2020WordCraft: An Environment for Benchmarking Commonsense Agentscited by 14
  21. 2017Weakly-supervised Semantic Parsing with Abstract Examplescited by 13
  22. 2018Cascaded Mutual Modulation for Visual Reasoningcited by 5
  23. 2018Working Memory Networks: Augmenting Memory Networks with a Relational Reasoning Modulecited by 3
  24. 2021Enforcing Consistency in Weakly Supervised Semantic Parsingcited by 3
  25. 2018Object Ordering with Bidirectional Matchings for Visual Reasoningcited by 2

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1; 2025: 1.

3 papers.

  1. 2023On the Effectiveness of Offline RL for Dialogue Response Generationcited by 6
  2. 2024SEER: Facilitating Structured Reasoning and Explanation via Reinforcement Learningcited by 3
  3. 2025Knot So Simple: A Minimalistic Environment for Spatial Reasoningcited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025Reinforcement Learning for Large Model: A Survey
  2. 2025Reasoning Under 1 Billion: Memory-Augmented Reinforcement Learning for Large Language Models
  3. 2025Reward Evolution with Graph-of-Thoughts: A Bi-Level Language Model Framework for Reinforcement Learning
  4. 2026Long-Horizon Language Model Reinforcement Learning via Progressive Point Matching
  5. 2025Open Vision Reasoner: Transferring Linguistic Cognitive Behavior for Visual Reasoning
  6. 2026A Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and Solutions
  7. 2026Nanbeige4.1-3B: A Small General Model that Reasons, Aligns, and Acts
  8. 2025Learning Only with Images: Visual Reinforcement Learning with Reasoning, Rendering, and Visual Feedback