mlsanityData to 7 Oct 2026

HALMA: Humanlike Abstraction Learning Meets Affordance in Rapid Problem Solving

arXiv 2102.11344 ↗First posted 22 Feb 2021cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

Humans learn compositional and causal abstraction, ie, knowledge, in response to the structure of naturalistic tasks. When presented with a problem-solving task involving some objects, toddlers would first interact with these objects to reckon what they are and what can be done with them. Leveraging these concepts, they could understand the internal structure of this task, without seeing all of the problem instances. Remarkably, they further build cognitively executable strategies to rapidly solve novel problems. To empower a learning agent with similar capability, we argue there shall be three levels of generalization in how an agent represents its knowledge: perceptual, conceptual, and algorithmic. In this paper, we devise the very first systematic benchmark that offers joint evaluation covering all three levels. This benchmark is centered around a novel task domain, HALMA, for visual concept development and rapid problem-solving. Uniquely, HALMA has a minimum yet complete concept space, upon which we introduce a novel paradigm to rigorously diagnose and dissect learning agents' capability in understanding and generalizing complex and structural concepts. We conduct extensive experiments on reinforcement learning agents with various inductive biases and carefully report their proficiency and weakness.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 33 of its 72 references that are in the map. First posted on arXiv 22 Feb 2021.

Builds on

33 of 72 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2016Layer Normalizationcited by 5,579
  4. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  5. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  6. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  7. 2018Relational inductive biases, deep learning, and graph networkscited by 1,341
  8. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  9. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  10. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
  11. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 388
  12. 2016Learning to reinforcement learncited by 369
21 more
  1. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 359
  2. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 352
  3. 2018Quantifying Generalization in Reinforcement Learningcited by 345
  4. 2017StarCraft II: A New Challenge for Reinforcement Learningcited by 268
  5. 2017Emergence of Grounded Compositional Language in Multi-Agent Populationscited by 257
  6. 2019RAVEN: A Dataset for Relational and Analogical Visual rEasoNingcited by 225
  7. 2016Attend, Infer, Repeat: Fast Scene Understanding with Generative Modelscited by 209
  8. 2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 204
  9. 2016Learning to Navigate in Complex Environmentscited by 192
  10. 2020SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decompositioncited by 190
  11. 2018Measuring abstract reasoning in neural networkscited by 165
  12. 2016Value Iteration Networkscited by 158
  13. 2018Visual Semantic Navigation using Scene Priorscited by 127
  14. 2018Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Inputcited by 102
  15. 2019An investigation of model-free planningcited by 64
  16. 2020Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sensecited by 62
  17. 2017Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physicscited by 57
  18. 2018Unsupervised Predictive Memory in a Goal-Directed Agentcited by 54
  19. 2020What can I do here? A Theory of Affordances in Reinforcement Learningcited by 33
  20. 2018Been There, Done That: Meta-Learning with Episodic Recallcited by 25
  21. 2019Embodied Multimodal Multitask Learningcited by 12

Built on by

Cited by year

Citing papers per year: 2021: 4; 2022: 6; 2023: 5.

15 papers.

Sort:

By area:

  1. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  2. 2021Abstract Spatial-Temporal Reasoning via Probabilistic Abduction and Executioncited by 43
  3. 2022Bongard-HOI: Benchmarking Few-Shot Visual Reasoning for Human-Object Interactionscited by 34
  4. 2021ACRE: Abstract Causal REasoning Beyond Covariationcited by 30
  5. 2023SparseDFF: Sparse-View Feature Distillation for One-Shot Dexterous Manipulationcited by 22
  6. 2023Bongard-OpenWorld: Few-Shot Reasoning for Free-form Visual Concepts in the Real Worldcited by 18
  7. 2023MEWL: Few-shot multimodal word learning with referential uncertaintycited by 17
  8. 2023I-PHYRE: Interactive Physical Reasoningcited by 15
  9. 2022RelViT: Concept-guided Vision Transformer for Visual Relational Reasoningcited by 14
  10. 2022Neural-Symbolic Recursive Machine for Systematic Generalizationcited by 12
  11. 2022Interactive Visual Reasoning under Uncertaintycited by 8
  12. 2023Active Reasoning in an Open-World Environmentcited by 8
3 more
  1. 2021A Minimalist Dataset for Systematic Generalization of Perception, Syntax, and Semanticscited by 6
  2. 2022On the Complexity of Bayesian Generalizationcited by 3
  3. 2022AIGenC: An AI generalisation model via creativitycited by 0