mlsanityData to 14 Oct 2026

Human-like compositional learning of visually-grounded concepts using synthetic environments

arXiv 2504.06618 ↗First posted 9 Apr 2025cs.CVcited by 0

On the map: Deep reinforcement learning methods → RL training of language and agent models

Abstract (arXiv)

The compositional structure of language enables humans to decompose complex phrases and map them to novel visual concepts, showcasing flexible intelligence. While several algorithms exhibit compositionality, they fail to elucidate how humans learn to compose concept classes and ground visual cues through trial and error. To investigate this multi-modal learning challenge, we designed a 3D synthetic environment in which an agent learns, via reinforcement, to navigate to a target specified by a natural language instruction. These instructions comprise nouns, attributes, and critically, determiners, prepositions, or both. The vast array of word combinations heightens the compositional complexity of the visual grounding task, as navigating to a blue cube above red spheres is not rewarded when the instruction specifies navigating to "some blue cubes below the red sphere". We first demonstrate that reinforcement learning agents can ground determiner concepts to visual targets but struggle with more complex prepositional concepts. Second, we show that curriculum learning, a strategy humans employ, enhances concept learning efficiency, reducing the required training episodes by 15% in determiner environments and enabling agents to easily learn prepositional concepts. Finally, we establish that agents trained on determiner or prepositional concepts can decompose held-out test instructions and rapidly adapt their navigation policies to unseen visual object combinations. Leveraging synthetic environments, our findings demonstrate that multi-modal reinforcement learning agents can achieve compositional understanding of complex concept classes and highlight the efficacy of human-like learning strategies in improving artificial systems' learning efficiency.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods. Builds on 32 of its 52 references that are in the map. First posted on arXiv 9 Apr 2025.

Builds on

32 of 52 references.

References · 32

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Segment Anythingcited by 8,234
  3. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  4. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  5. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  6. 2022When and why vision-language models behave like bags-of-words, and what to do about it?cited by 610
  7. 2022Visual Programming: Compositional visual reasoning without trainingcited by 516
  8. 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 439
  9. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
  10. 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 356
  11. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 305
  12. 2021Curriculum Learning: A Surveycited by 259
20 more
  1. 2023What's "up" with vision-language models? Investigating their struggle with spatial reasoningcited by 254
  2. 2023Teaching CLIP to Count to Tencited by 198
  3. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 182
  4. 2022CREPE: Can Vision-Language Foundation Models Reason Compositionally?cited by 181
  5. 2019Task-Driven Modular Networks for Zero-Shot Compositional Learningcited by 121
  6. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 117
  7. 2022Does CLIP Bind Concepts? Probing Compositionality in Large Image Modelscited by 102
  8. 2023Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Taskcited by 93
  9. 2023Scaling Vision-Language Models with Sparse Mixture of Expertscited by 88
  10. 2023GALIP: Generative Adversarial CLIPs for Text-to-Image Synthesiscited by 85
  11. 2023CLIP-Count: Towards Text-Guided Zero-Shot Object Countingcited by 83
  12. 2020Referring Expression Comprehension: A Survey of Methods and Datasetscited by 79
  13. 2020Compositional Learning of Image-Text Query for Image Retrievalcited by 57
  14. 2017Teaching Compositionality to CNNscited by 18
  15. 2021Zero-Shot Compositional Concept Learningcited by 16
  16. 2021One-shot learning of paired association navigation with biologically plausible schemascited by 3
  17. 2021A nonlinear hidden layer enables actor-critic agents to learn multiple paired association navigationcited by 3
  18. 2023DetermiNet: A Large-Scale Diagnostic Dataset for Complex Visually-Grounded Referencing using Determinerscited by 3
  19. 2023STUPD: A Synthetic Dataset for Spatial and Temporal Relation Reasoningcited by 3
  20. 2023Compositional Learning of Visually-Grounded Concepts Using Reinforcementcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2020Zero-Shot Compositional Policy Learning via Language Grounding
  2. 2025Compositional Instruction Following with Language Models and Reinforcement Learning
  3. 2024Iterated Learning Improves Compositionality in Large Vision-Language Models
  4. 2023Im-Promptu: In-Context Composition from Image Prompts
  5. 2026Compositional Reasoning in Language Models under Reinforcement Learning Post-Training
  6. 2025Unveiling the Compositional Ability Gap in Vision-Language Reasoning Model
  7. 2025Temporal Representation Alignment: Successor Features Enable Emergent Compositionality in Robot Instruction Following
  8. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learning