# Facilitating Video Story Interaction with Multi-Agent Collaborative System

arXiv: https://arxiv.org/abs/2505.03807 · HTML: https://mlsanity.com/paper/2505.03807/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 May 2025.

## Built on by (most cited first, 2 of 2)

- [Vidmento: Creating Video Stories Through Context-Aware Expansion With Generative Video](https://mlsanity.com/paper/2601.22013/) (2026)
- [Seeing My Future: Predicting Situated Interaction Behavior in Virtual Reality](https://mlsanity.com/paper/2510.10742/) (2025)

## Abstract (arXiv)

Video story interaction enables viewers to engage with and explore narrative content for personalized experiences. However, existing methods are limited to user selection, specially designed narratives, and lack customization. To address this, we propose an interactive system based on user intent. Our system uses a Vision Language Model (VLM) to enable machines to understand video stories, combining Retrieval-Augmented Generation (RAG) and a Multi-Agent System (MAS) to create evolving characters and scene experiences. It includes three stages: 1) Video story processing, utilizing VLM and prior knowledge to simulate human understanding of stories across three modalities. 2) Multi-space chat, creating growth-oriented characters through MAS interactions based on user queries and story stages. 3) Scene customization, expanding and visualizing various story scenes mentioned in dialogue. Applied to the Harry Potter series, our study shows the system effectively portrays emergent character social behavior and growth, enhancing the interactive experience in the video story world.