What Comes Next? Omni-StoryBench for Evaluating Story-Grounded Omnimodal Generation
Not on the map. Why?
Abstract (arXiv)
Omnimodal evaluation should go beyond independent text, image, and speech production: individually plausible outputs may not express a coherent shared event. We introduce Omni-StoryBench, a story-grounded omnimodal benchmark evaluating whether models can coherently continue stories across image, narration, and speech. Each instance provides a current storybook page and structured next-page conditions, requiring models to generate the next illustration, narration, and spoken character utterance. Omni-StoryBench contains 900 rigorously validated story transitions from openly licensed children's books, with ground-truth next-page references and speech metadata. We evaluate systems with modality-specific metrics and consistency-centered LLM-as-a-judge rubrics for context preservation, condition following, reference consistency, and cross-modal coherence. Across 32 baseline configurations spanning orchestration, semi-orchestration, and native any-to-any paradigms, we find orchestration with strong VLM planning most reliable, while current native omnimodal models often struggle with output completeness and controllability. Our analysis shows text-side performance is associated with image and speech quality, but image generation and visual continuity form the clearest observed bottleneck among the evaluated configurations. These results position Omni-StoryBench as a system-level benchmark measuring coherent omnimodal generation beyond isolated modality quality.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 42 of its 81 references that are in the map. First posted on arXiv 29 Sep 2026.
Builds on
42 of 81 references.
References · 42
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2025Qwen3 Technical Reportcited by 8,820
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2024GPT-4o System Cardcited by 4,588
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
- 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 940
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
30 more
- 2024Emu3: Next-Token Prediction is All You Needcited by 781
- 2025Qwen2.5-Omni Technical Reportcited by 736
- 2024Movie Gen: A Cast of Media Foundation Modelscited by 644
- 2025Qwen3-Omni Technical Reportcited by 497
- 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 466
- 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 295
- 2026Gemma 4 Technical Reportcited by 259
- 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 250
- 2025Kimi-Audio Technical Reportcited by 222
- 2025Llama-Nemotron: Efficient Reasoning Modelscited by 158
- 2024SEED-Story: Multimodal Long Story Generation with Large Language Modelcited by 73
- 2024EMOVA: Empowering Language Models to See, Hear and Speak with Vivid Emotionscited by 45
- 2025VoxCPM: Tokenizer-Free TTS for Context-Aware Speech Generation and True-to-Life Voice Cloningcited by 42
- 2024OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generationcited by 31
- 2026Nemotron 3 Nano Omni: Efficient and Open Multimodal Intelligencecited by 30
- 2025Uni-MMMU: A Massive Multi-discipline Multimodal Unified Benchmarkcited by 29
- 2024Holistic Evaluation for Interleaved Text-and-Image Generationcited by 28
- 2024Benchmarking Large Multimodal Models against Common Corruptionscited by 24
- 2025NExT-OMNI: Towards Any-to-Any Omnimodal Foundation Models with Discrete Flow Matchingcited by 21
- 2024Cross-Modal Consistency in Multimodal Large Language Modelscited by 19
- 2026Audio Flamingo Next: Next-Generation Open Audio-Language Models for Speech, Sound, and Musiccited by 19
- 2026Omni-Diffusion: Unified Multimodal Understanding and Generation with Masked Discrete Diffusioncited by 13
- 2026UniM: A Unified Any-to-Any Interleaved Multimodal Benchmarkcited by 11
- 2026MOSS-Audio Technical Reportcited by 11
- 2025MMMG: a Comprehensive and Reliable Benchmark for Multitask Multimodal Generationcited by 6
- 2024MathBridge: A Large Corpus Dataset for Translating Spoken Mathematical Expressions into LaTeX Formulas for Improved Readabilitycited by 5
- 2026Dynin-Omni: Omnimodal Unified Large Diffusion Language Modelcited by 5
- 2026HyperCLOVA X 8B Omnicited by 4
- 2025Node-Based Editing for Multimodal Generation of Text, Audio, Image, and Videocited by 3
- 2026EXAONE 4.5 Technical Reportcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024OmniBench: Towards The Future of Universal Omni-Language Models
- 2025SeqBench: Benchmarking Sequential Narrative Generation in Text-to-Video Models
- 2024Movie2Story: A framework for understanding videos and telling stories in the form of novel text
- 2025Story2Board: A Training-Free Approach for Expressive Storyboard Generation
- 2023Intelligent Grimm -- Open-ended Visual Storytelling via Latent Diffusion Models
- 2026StoryAlign: Evaluating and Training Reward Models for Story Generation
- 2026Can MiniMax-H3 Reason About the Physical World? An Evaluation of Omni-Modal Generative Model
- 2026StoryState: Agent-Based State Control for Consistent and Editable Storybooks