mlsanityData to 7 Oct 2026

Mitigating Hallucination in Fictional Character Role-Play

arXiv 2406.17260 ↗First posted 25 Jun 2024cs.CLcited by 17

Not on the map. Why?

Abstract (arXiv)

Role-playing has wide-ranging applications in customer support, embodied agents, and computational social science. The influence of parametric world knowledge of large language models (LLMs) often causes role-playing characters to act out of character and to hallucinate about things outside the scope of their knowledge. In this work, we focus on the evaluation and mitigation of hallucination in fictional character role-play. We introduce a dataset with over 2,000 characters and 72,000 interviews, including 18,000 adversarial questions. We propose RoleFact, a role-playing method that mitigates hallucination by modulating the influence of parametric knowledge using a pre-calibrated confidence threshold. Experiments show that the proposed method improves the factual precision of generated responses by 18% for adversarial questions with a 44% reduction in temporal hallucination for time-sensitive interviews. The code and the dataset are available at https://github.com/NafisSadeq/rolefact.git.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 35 of its 51 references that are in the map. First posted on arXiv 25 Jun 2024.

Builds on

35 of 51 references.

References · 35

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  7. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  8. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  9. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  10. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  11. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  12. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
23 more
  1. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  2. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  3. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  4. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  5. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 820
  6. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  7. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  8. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 350
  9. 2023Character-LLM: A Trainable Agent for Role-Playingcited by 335
  10. 2023InCharacter: Evaluating Personality Fidelity in Role-Playing Agents through Psychological Interviewscited by 177
  11. 2017Faithful to the Original: Fact Aware Neural Abstractive Summarizationcited by 141
  12. 2023ExpertPrompting: Instructing Large Language Models to be Distinguished Expertscited by 140
  13. 2021Q2: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answeringcited by 131
  14. 2021Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Groundingcited by 106
  15. 2024Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-Alignmentcited by 104
  16. 2023Aligning Language Models to User Opinionscited by 98
  17. 2023Mitigating Large Language Model Hallucinations via Autonomous Knowledge Graph-based Retrofittingcited by 70
  18. 2020Multi-Fact Correction in Abstractive Text Summarizationcited by 66
  19. 2023CharacterChat: Learning towards Conversational AI with Personalized Social Supportcited by 58
  20. 2023Knowledge Card: Filling LLMs' Knowledge Gaps with Plug-in Specialized Language Modelscited by 46
  21. 2022Meet Your Favorite Character: Open-domain Chatbot Mimicking Fictional Characters with only a Few Utterancescited by 32
  22. 2022Large Language Models Meet Harry Potter: A Bilingual Dataset for Aligning Dialogue Agents with Characterscited by 16
  23. 2022MBTI Personality Prediction for Fictional Characters Using Movie Scriptscited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 7; 2026: 6.

17 papers.

Sort:

By area:

  1. 2024The Oscars of AI Theater: A Survey on Role-Playing with Language Modelscited by 47
  2. 2025Enhancing Persona Consistency for LLMs' Role-Playing using Persona-Aware Contrastive Learningcited by 27
  3. 2024RoleBreak: Character Hallucination as a Jailbreak Attack in Role-Playing Systemscited by 19
  4. 2025SOTOPIA-Omega: Dynamic Strategy Injection Learning and Social Instruction Following Evaluation for Social Agentscited by 16
  5. 2025OmniCharacter: Towards Immersive Role-Playing Agents with Seamless Speech-Language Personality Interactioncited by 14
  6. 2026A Survey on LLM-based Conversational User Simulationcited by 14
  7. 2025Automated Privacy Information Annotation in Large Language Model Interactionscited by 6
  8. 2024SHARP: Unlocking Interactive Hallucination via Stance Transfer in Role-Playing LLMscited by 5
  9. 2025Codifying Character Logic in Role-Playingcited by 5
  10. 2025SpeechRole: A Large-Scale Dataset and Benchmark for Evaluating Speech Role-Playing Agentscited by 4
  11. 2024Tell Me What You Don't Know: Enhancing Refusal Capabilities of Role-Playing Agents via Representation Space Analysis and Editingcited by 3
  12. 2026RoleCDE:Benchmarking and Mitigating Role-Alignment Trade-offs in Role-Playing Agentscited by 3
5 more
  1. 2025Learning to Search Effective Example Sequences for In-Context Learningcited by 2
  2. 2026Identifying and Mitigating Bottlenecks in Role-Playing Agents: A Systematic Study of Disentangling Character Profile Axescited by 2
  3. 2026ArcANE: Do Role-Playing Language Agents Stay in Character at the Right Time?cited by 2
  4. 2026BOOKMARKS: Efficient Active Storyline Memory for Role-playingcited by 0
  5. 2026CHARM: Character Hallucination for Multicultural Role Play Benchmarkcited by 0