mlsanityData to 7 Oct 2026

How Do Hackathons Foster Creativity? Towards AI Collaborative Evaluation of Creativity at Scale

arXiv 2503.04290 ↗First posted 6 Mar 2025cs.HC cs.AI cs.SEcited by 0

Not on the map. Why?

Abstract (arXiv)

Hackathons have become popular collaborative events for accelerating the development of creative ideas and prototypes. There are several case studies showcasing creative outcomes across domains such as industry, education, and research. However, there are no large-scale studies on creativity in hackathons which can advance theory on how hackathon formats lead to creative outcomes. We conducted a computational analysis of 193,353 hackathon projects. By operationalizing creativity through usefulness and novelty, we refined our dataset to 10,363 projects, allowing us to analyze how participant characteristics, collaboration patterns, and hackathon setups influence the development of creative projects. The contribution of our paper is twofold: We identified means for organizers to foster creativity in hackathons. We also explore the use of large language models (LLMs) to augment the evaluation of creative outcomes and discuss challenges and opportunities of doing this, which has implications for creativity research at large.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 97 references that are in the map. First posted on arXiv 6 Mar 2025.

Builds on

25 of 97 references.

References · 25

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2023Mistral 7Bcited by 3,065
  5. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  6. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  7. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 849
  8. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 729
  9. 2023Prometheus: Inducing Fine-grained Evaluation Capability in Language Modelscited by 558
  10. 2019Quantifying the Carbon Emissions of Machine Learningcited by 516
  11. 2024RewardBench: Evaluating Reward Models for Language Modelingcited by 480
  12. 2024Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Modelscited by 431
13 more
  1. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 420
  2. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 416
  3. 2024Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Modelscited by 301
  4. 2023Exploring the Use of Large Language Models for Reference-Free Text Quality Evaluation: An Empirical Studycited by 85
  5. 2022Designing Creative AI Partners with COFI: A Framework for Modeling Interaction in Human-AI Co-Creative Systemscited by 59
  6. 2024ChatGPT Rates Natural Language Explanation Quality Like Humans: But on Which Scales?cited by 17
  7. 2024Antagonistic AIcited by 5
  8. 2022The Developers' Design Thinking Toolbox in Hackathons: A Study on the Recurring Design Methods in Software Development Marathonscited by 2
  9. 2017Hack Weeks as a model for Data Science Education and Collaborationcited by 1
  10. 2020How to organize an in-person, online or hybrid hackathon -- A revised planning kitcited by 1
  11. 2021The Secret Life of Hackathon Codecited by 1
  12. 2021Tracking Hackathon Code Creation and Reusecited by 1
  13. 2022One-off Events? An Empirical Study of Hackathon Code Creation and Reusecited by 1

Built on by

Not cited yet.