# Creative Problem Solving in Artificially Intelligent Agents: A Survey and Framework

arXiv: https://arxiv.org/abs/2204.10358 · HTML: https://mlsanity.com/paper/2204.10358/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 119 references that are in the map.
- First posted on arXiv 21 Apr 2022.

## Builds on (most cited first, 27 of 27)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Emergent Tool Use From Multi-Agent Autocurricula](https://mlsanity.com/paper/1909.07528/) (2019)
- [Neurosymbolic AI: The 3rd Wave](https://mlsanity.com/paper/2012.05876/) (2020)
- [Neural Style Transfer: A Review](https://mlsanity.com/paper/1705.04058/) (2017)
- [Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutions](https://mlsanity.com/paper/1901.01753/) (2019)
- [Neural Task Programming: Learning to Generalize Across Hierarchical Tasks](https://mlsanity.com/paper/1710.01813/) (2017)
- [Learning to Control Self-Assembling Morphologies: A Study of Generalization via Modularity](https://mlsanity.com/paper/1902.05546/) (2019)
- [KETO: Learning Keypoint Representations for Tool Manipulation](https://mlsanity.com/paper/1910.11977/) (2019)
- [Autocurricula and the Emergence of Innovation from Social Interaction: A Manifesto for Multi-Agent Intelligence Research](https://mlsanity.com/paper/1903.00742/) (2019)
- [Learning Symbolic Operators for Task and Motion Planning](https://mlsanity.com/paper/2103.00589/) (2021)
- [Reinforcement Learning for Improving Agent Design](https://mlsanity.com/paper/1810.03779/) (2018)
- [Jointly Learning to Construct and Control Agents using Deep Reinforcement Learning](https://mlsanity.com/paper/1801.01432/) (2018)
- [Structured agents for physical construction](https://mlsanity.com/paper/1904.03177/) (2019)
- [Improvisation through Physical Understanding: Using Novel Objects as Tools with Visual Foresight](https://mlsanity.com/paper/1904.05538/) (2019)
- [Generalization to New Actions in Reinforcement Learning](https://mlsanity.com/paper/2011.01928/) (2020)
- [SPOTTER: Extending Symbolic Planning Operators through Targeted Reinforcement Learning](https://mlsanity.com/paper/2012.13037/) (2020)
- [Tool Macgyvering: Tool Construction Using Geometric Reasoning](https://mlsanity.com/paper/1902.03666/) (2019)
- [Adapting Everyday Manipulation Skills to Varied Scenarios](https://mlsanity.com/paper/1803.02743/) (2018)
- [STRIPS Action Discovery](https://mlsanity.com/paper/2001.11457/) (2020)
- [Tool Macgyvering: A Novel Framework for Combining Tool Substitution and Construction](https://mlsanity.com/paper/2008.10638/) (2020)
- [Feature Guided Search for Creative Problem Solving Through Tool Construction](https://mlsanity.com/paper/2008.10685/) (2020)
- [Skill Learning by Autonomous Robotic Playing using Active Learning and Creativity](https://mlsanity.com/paper/1706.08560/) (2017)
- [Creative Invention Benchmark](https://mlsanity.com/paper/1805.03720/) (2018)
- [Tool Substitution with Shape and Material Reasoning Using Dual Neural Networks](https://mlsanity.com/paper/1911.04521/) (2019)
- [Perception-Informed Autonomous Environment Augmentation With Modular Robots](https://mlsanity.com/paper/1710.01840/) (2017)

## Built on by (most cited first, 4 of 4)

- [A Call for Collaborative Intelligence: Why Human-Agent Systems Should Precede AI Autonomy](https://mlsanity.com/paper/2506.09420/) (2025)
- [Creative Problem Solving in Large Language and Vision Models -- What Would it Take?](https://mlsanity.com/paper/2405.01453/) (2024)
- [Mentigo: An Intelligent Agent for Mentoring Students in the Creative Problem Solving Process](https://mlsanity.com/paper/2409.14228/) (2024)
- [Novelty Adaptation Through Hybrid Large Language Model (LLM)-Symbolic Planning and LLM-guided Reinforcement Learning](https://mlsanity.com/paper/2603.11351/) (2026)

## Abstract (arXiv)

Creative Problem Solving (CPS) is a sub-area within Artificial Intelligence (AI) that focuses on methods for solving off-nominal, or anomalous problems in autonomous systems. Despite many advancements in planning and learning, resolving novel problems or adapting existing knowledge to a new context, especially in cases where the environment may change in unpredictable ways post deployment, remains a limiting factor in the safe and useful integration of intelligent systems. The emergence of increasingly autonomous systems dictates the necessity for AI agents to deal with environmental uncertainty through creativity. To stimulate further research in CPS, we present a definition and a framework of CPS, which we adopt to categorize existing AI methods in this field. Our framework consists of four main components of a CPS problem, namely, 1) problem formulation, 2) knowledge representation, 3) method of knowledge manipulation, and 4) method of evaluation. We conclude our survey with open research questions, and suggested directions for the future.