Does Less Hallucination Mean Less Creativity? An Empirical Investigation in LLMs
Not on the map. Why?
Abstract (arXiv)
Large Language Models (LLMs) exhibit remarkable capabilities in natural language understanding and reasoning, but suffer from hallucination: the generation of factually incorrect content. While numerous methods have been developed to reduce hallucinations, their impact on creative generations remains unexplored. This gap is particularly critical for AI-assisted scientific discovery, which requires both factual accuracy and creative hypothesis generation. We investigate how three hallucination-reduction techniques: Chain of Verification (CoVe), Decoding by Contrasting Layers (DoLa), and Retrieval-Augmented Generation (RAG), affect creativity in LLMs. Evaluating multiple model families (LLaMA, Qwen, Mistral) at varying scales (1B - 70B parameters) on two creativity benchmarks (NeoCoder and CS4), we find that these methods have opposing effects on divergent creativity. CoVe enhances divergent thinking, DoLa suppresses it, and RAG shows minimal impact. Our findings provide guidance for selecting appropriate hallucination-reduction methods in scientific applications, where the balance between factual accuracy and creative exploration is crucial.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 26 references that are in the map. First posted on arXiv 12 Dec 2025.
Builds on
15 of 26 references.
References · 15
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
- 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
- 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
- 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 476
- 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 376
- 2023LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examplescited by 177
- 2024CodeRAG-Bench: Can Retrieval Augment Code Generation?cited by 91
- 2024Benchmarking Language Model Creativity: A Case Study on Code Generationcited by 40
- 2025ParaThinker: Native Parallel Thinking as a New Paradigm to Scale LLM Test-time Computecited by 38
- 2025The Blessing and Curse of Dimensionality in Safety Alignmentcited by 10
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Think More, Hallucinate Less: Mitigating Hallucinations via Dual Process of Fast and Slow Thinking
- 2024Interpreting and Mitigating Hallucination in MLLMs through Multi-agent Debate
- 2025Triggering Hallucinations in LLMs: A Quantitative Study of Prompt-Induced Hallucination in Large Language Models
- 2025Auditing Meta-Cognitive Hallucinations in Reasoning Large Language Models
- 2025From "Hallucination" to "Suture": Insights from Language Philosophy to Enhance Large Language Models
- 2024Banishing LLM Hallucinations Requires Rethinking Generalization
- 2026PRISM: Probing Reasoning, Instruction, and Source Memory in LLM Hallucinations
- 2025Smoothing Out Hallucinations: Mitigating LLM Hallucination with Smoothed Knowledge Distillation