mlsanityData to 7 Oct 2026

Kahaani: A Multimodal Co-Creative Storytelling System

arXiv 2409.11261 ↗First posted 17 Sep 2024cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

This paper introduces Kahaani, a multimodal, co-creative storytelling system that leverages Generative Artificial Intelligence, designed for children to address the challenge of sustaining engagement to foster educational narrative experiences. Here we define co-creative as a collaborative creative process in which both the child and Kahaani contribute to the generation of the story. The system combines Large Language Model (LLM), Text-to-Speech (TTS), Text-to-Music (TTM), and Text-to-Video (TTV) generation to produce a rich, immersive, and accessible storytelling experience. The system grounds the co-creation process in two classical storytelling framework, Freytag's Pyramid and Propp's Narrative Functions. The main goals of Kahaani are: (1) to help children improve their English skills, (2) to teach important life lessons through story morals, and (3) to help them understand how stories are structured, all in a fun and engaging way. We present evaluations for each AI component used, along with a user study involving three parent-child pairs to assess the overall experience and educational value of the system.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 12 of its 49 references that are in the map. First posted on arXiv 17 Sep 2024.

Builds on

12 of 49 references.

References · 12

  1. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  2. 2018Hierarchical Neural Story Generationcited by 1,182
  3. 2018Plan-And-Write: Towards Better Automatic Storytellingcited by 232
  4. 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 148
  5. 2023FETV: A Benchmark for Fine-Grained Evaluation of Open-Domain Text-to-Video Generationcited by 119
  6. 2020PlotMachines: Outline-Conditioned Generation with Dynamic Plot State Trackingcited by 104
  7. 2024Evaluation of Text-to-Video Generation Models: A Dynamics Perspectivecited by 66
  8. 2018Story Ending Generation with Incremental Encoding and Commonsense Knowledgecited by 60
  9. 2021A Plug-and-Play Method for Controlled Text Generationcited by 55
  10. 2024Towards A Better Metric for Text-to-Video Generationcited by 50
  11. 2021Assessing Evaluation Metrics for Speech-to-Speech Translationcited by 10
  12. 2023Conveying the Predicted Future to Users: A Case Study of Story Plot Predictioncited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.