# IDEAFix: Evaluation Framework for Creative Defixation Prompting in LLMs

arXiv: https://arxiv.org/abs/2606.00875 · HTML: https://mlsanity.com/paper/2606.00875/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 62 references that are in the map.
- First posted on arXiv 30 May 2026.

## Builds on (most cited first, 24 of 24)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Automatic Chain of Thought Prompting in Large Language Models](https://mlsanity.com/paper/2210.03493/) (2022)
- [Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMs](https://mlsanity.com/paper/2308.13387/) (2023)
- [Generative artificial intelligence enhances creativity but reduces the diversity of novel content](https://mlsanity.com/paper/2312.00506/) (2023)
- [Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)](https://mlsanity.com/paper/2510.22954/) (2025)
- [Homogenization Effects of Large Language Models on Human Creative Ideation](https://mlsanity.com/paper/2402.01536/) (2024)
- [Is Temperature the Creativity Parameter of Large Language Models?](https://mlsanity.com/paper/2405.00492/) (2024)
- [On the Creativity of Large Language Models](https://mlsanity.com/paper/2304.00008/) (2023)
- [Base Models Beat Aligned Models at Randomness and Creativity](https://mlsanity.com/paper/2505.00047/) (2025)
- [Confabulation: The Surprising Value of Large Language Model Hallucinations](https://mlsanity.com/paper/2406.04175/) (2024)
- [LatEval: An Interactive LLMs Evaluation Benchmark with Incomplete Information from Lateral Thinking Puzzles](https://mlsanity.com/paper/2308.10855/) (2023)
- [Towards Scientific Discovery with Generative AI: Progress, Opportunities, and Challenges](https://mlsanity.com/paper/2412.11427/) (2024)
- [The Effects of Generative AI on Design Fixation and Divergent Thinking](https://mlsanity.com/paper/2403.11164/) (2024)
- [MacGyver: Are Large Language Models Creative Problem Solvers?](https://mlsanity.com/paper/2311.09682/) (2023)
- [Divergent Creativity in Humans and Large Language Models](https://mlsanity.com/paper/2405.13012/) (2024)
- [Automated Creativity Evaluation for Large Language Models: A Reference-Based Approach](https://mlsanity.com/paper/2504.15784/) (2025)
- [AutoTRIZ: Automating Engineering Innovation with TRIZ and Large Language Models](https://mlsanity.com/paper/2403.13002/) (2024)
- [Creativity in AI: Progresses and Challenges](https://mlsanity.com/paper/2410.17218/) (2024)
- [Do LLMs Agree on the Creativity Evaluation of Alternative Uses?](https://mlsanity.com/paper/2411.15560/) (2024)
- [Has the Creativity of Large-Language Models peaked? An analysis of inter- and intra-LLM variability](https://mlsanity.com/paper/2504.12320/) (2025)

## Abstract (arXiv)

Large language models (LLMs) are increasingly used for tasks involving creative problem solving and idea generation. However, there is a lack of consensus concerning their creative capabilities: some studies report superior performances compared to humans, while others highlight structural limitations such as fixation and the homogenization of outputs. Existing evaluation approaches either rely on narrow, decontextualized tasks that do not capture goal-oriented generation or on broader settings that confound multiple aspects of the creative process, making it difficult to isolate the effects of task formulation, prompting, and evaluation design. Significantly, the role of structured prompting strategies in shaping idea generation remains underexplored. Therefore, we introduce IDEAFix, an evaluation framework for analyzing divergent thinking in open-ended idea generation tasks. We prompt models to generate multiple original solutions to controlled variations of short design scenarios, task attributes, and defixation prompting strategies. This design enables systematic analysis of how structured guidance influences LLMs' idea generation. Our results show that both task formulation and attribute selection significantly affect models' performance, and that simple prompting strategies can boost the originality of solutions. However, we also observe persistent output homogenization across models, confirming inherent limits in their ability to generate diverse solutions. Overall, IDEAFix provides a controlled, extensible framework for studying the mechanisms underlying LLMs' creativity.