mlsanityData to 14 Oct 2026

GENIE: A Fine-Grained Measure for Novelty

arXiv 2606.12790 ↗First posted 11 Jun 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Large Language Models have consistently demonstrated a lack of creativity and diversity across tasks. Prior work has focused on addressing whether models are capable of generating creative outputs. Here, we aim to consider novelty and investigate what makes model-generated content novel or not novel in a task-specific manner. We propose a fine-grained evaluation metric GENIE to measure the novelty of responses along task-specific features with respect to a population of responses. We show that unlike GENIE, holistic metrics struggle to capture the high-dimensionality of novelty and do not provide insight on which properties they target. Finally, we use GENIE to measure the effectiveness of mitigation methods that address creativity to better understand where these methods can improve novelty.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 48 references that are in the map. First posted on arXiv 11 Jun 2026.

Builds on

30 of 48 references.

References · 30

  1. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  2. 2018Hierarchical Neural Story Generationcited by 1,193
  3. 2023Understanding the Effects of RLHF on LLM Generalisation and Diversitycited by 396
  4. 2024Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Expertscited by 376
  5. 2023Does Writing with Language Models Reduce Content Diversity?cited by 192
  6. 2023Art or Artifice? Large Language Models and the False Promise of Creativitycited by 172
  7. 2025Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)cited by 141
  8. 2025Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversitycited by 105
  9. 2024Agents' Room: Narrative Generation through Multi-step Collaborationcited by 57
  10. 2025Base Models Beat Aligned Models at Randomness and Creativitycited by 51
  11. 2024Can AI writing be salvaged? Mitigating Idiosyncrasies and Improving Human-AI Alignment in the Writing Process through Editscited by 49
  12. 2024Echoes in AI: Quantifying lack of plot diversity in LLM outputscited by 48
18 more
  1. 2024Detection and Measurement of Syntactic Templates in Generated Textcited by 44
  2. 2025We're Different, We're the Same: Creative Homogeneity Across LLMscited by 42
  3. 2025LitBench: A Benchmark and Dataset for Reliable Evaluation of Creative Writingcited by 42
  4. 2024Exploring Precision and Recall to assess the quality and diversity of LLMscited by 29
  5. 2024Evaluating n-Gram Novelty of Language Models Using Rusty-DAWGcited by 25
  6. 2025AI-Slop to AI-Polish? Aligning Language Models through Edit-Based Writing Rewards and Test-time Computationcited by 25
  7. 2025Creative Preference Optimizationcited by 25
  8. 2025Diverse Preference Learning for Capabilities and Alignmentcited by 25
  9. 2024Detecting Mode Collapse in Language Models via Narrationcited by 24
  10. 2025QUDsim: Quantifying Discourse Similarities in LLM-Generated Textcited by 13
  11. 2025NarraBench: A Comprehensive Framework for Narrative Benchmarkingcited by 13
  12. 2025Evaluating Text Creativity across Diverse Domains: A Dataset and Large Language Model Evaluatorcited by 10
  13. 2026StoryScope: Investigating idiosyncrasies in AI fictioncited by 9
  14. 2026LLMs Corrupt Your Documents When You Delegatecited by 9
  15. 2025Frankentext: Stitching random text fragments into long-form narrativescited by 8
  16. 2025Measuring Combinatorial Creativity in Large Language Models and Humanscited by 7
  17. 2026CREATE: Testing LLMs for Associative Creativitycited by 7
  18. 2025Measuring LLM Novelty As The Frontier Of Original And High-Quality Outputcited by 6

Built on by

Not cited yet.