mlsanityData to 14 Oct 2026

What Comes Next? Omni-StoryBench for Evaluating Story-Grounded Omnimodal Generation

arXiv 2609.37317 ↗First posted 29 Sep 2026cs.CV cs.MMcited by 0

Not on the map. Why?

Abstract (arXiv)

Omnimodal evaluation should go beyond independent text, image, and speech production: individually plausible outputs may not express a coherent shared event. We introduce Omni-StoryBench, a story-grounded omnimodal benchmark evaluating whether models can coherently continue stories across image, narration, and speech. Each instance provides a current storybook page and structured next-page conditions, requiring models to generate the next illustration, narration, and spoken character utterance. Omni-StoryBench contains 900 rigorously validated story transitions from openly licensed children's books, with ground-truth next-page references and speech metadata. We evaluate systems with modality-specific metrics and consistency-centered LLM-as-a-judge rubrics for context preservation, condition following, reference consistency, and cross-modal coherence. Across 32 baseline configurations spanning orchestration, semi-orchestration, and native any-to-any paradigms, we find orchestration with strong VLM planning most reliable, while current native omnimodal models often struggle with output completeness and controllability. Our analysis shows text-side performance is associated with image and speech quality, but image generation and visual continuity form the clearest observed bottleneck among the evaluated configurations. These results position Omni-StoryBench as a system-level benchmark measuring coherent omnimodal generation beyond isolated modality quality.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 42 of its 81 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

42 of 81 references.

References · 42

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  4. 2024Qwen2.5 Technical Reportcited by 4,942
  5. 2024GPT-4o System Cardcited by 4,588
  6. 2025Qwen3-VL Technical Reportcited by 2,793
  7. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  8. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  9. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
  10. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 940
  11. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  12. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
30 more
  1. 2024Emu3: Next-Token Prediction is All You Needcited by 781
  2. 2025Qwen2.5-Omni Technical Reportcited by 736
  3. 2024Movie Gen: A Cast of Media Foundation Modelscited by 644
  4. 2025Qwen3-Omni Technical Reportcited by 497
  5. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 466
  6. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 295
  7. 2026Gemma 4 Technical Reportcited by 259
  8. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 250
  9. 2025Kimi-Audio Technical Reportcited by 222
  10. 2025Llama-Nemotron: Efficient Reasoning Modelscited by 158
  11. 2024SEED-Story: Multimodal Long Story Generation with Large Language Modelcited by 73
  12. 2024EMOVA: Empowering Language Models to See, Hear and Speak with Vivid Emotionscited by 45
  13. 2025VoxCPM: Tokenizer-Free TTS for Context-Aware Speech Generation and True-to-Life Voice Cloningcited by 42
  14. 2024OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generationcited by 31
  15. 2026Nemotron 3 Nano Omni: Efficient and Open Multimodal Intelligencecited by 30
  16. 2025Uni-MMMU: A Massive Multi-discipline Multimodal Unified Benchmarkcited by 29
  17. 2024Holistic Evaluation for Interleaved Text-and-Image Generationcited by 28
  18. 2024Benchmarking Large Multimodal Models against Common Corruptionscited by 24
  19. 2025NExT-OMNI: Towards Any-to-Any Omnimodal Foundation Models with Discrete Flow Matchingcited by 21
  20. 2024Cross-Modal Consistency in Multimodal Large Language Modelscited by 19
  21. 2026Audio Flamingo Next: Next-Generation Open Audio-Language Models for Speech, Sound, and Musiccited by 19
  22. 2026Omni-Diffusion: Unified Multimodal Understanding and Generation with Masked Discrete Diffusioncited by 13
  23. 2026UniM: A Unified Any-to-Any Interleaved Multimodal Benchmarkcited by 11
  24. 2026MOSS-Audio Technical Reportcited by 11
  25. 2025MMMG: a Comprehensive and Reliable Benchmark for Multitask Multimodal Generationcited by 6
  26. 2024MathBridge: A Large Corpus Dataset for Translating Spoken Mathematical Expressions into LaTeX Formulas for Improved Readabilitycited by 5
  27. 2026Dynin-Omni: Omnimodal Unified Large Diffusion Language Modelcited by 5
  28. 2026HyperCLOVA X 8B Omnicited by 4
  29. 2025Node-Based Editing for Multimodal Generation of Text, Audio, Image, and Videocited by 3
  30. 2026EXAONE 4.5 Technical Reportcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024OmniBench: Towards The Future of Universal Omni-Language Models
  2. 2025SeqBench: Benchmarking Sequential Narrative Generation in Text-to-Video Models
  3. 2024Movie2Story: A framework for understanding videos and telling stories in the form of novel text
  4. 2025Story2Board: A Training-Free Approach for Expressive Storyboard Generation
  5. 2023Intelligent Grimm -- Open-ended Visual Storytelling via Latent Diffusion Models
  6. 2026StoryAlign: Evaluating and Training Reward Models for Story Generation
  7. 2026Can MiniMax-H3 Reason About the Physical World? An Evaluation of Omni-Modal Generative Model
  8. 2026StoryState: Agent-Based State Control for Consistent and Editable Storybooks