See Before You Code: Learning Visual Priors for Spatially Aware Educational Animation Generation
Not on the map. Why?
Abstract (arXiv)
Large language models can generate executable code for educational animations, but the resulting renders often exhibit visual defects, including element overlap, misalignment, and broken animation continuity. These defects cannot be reliably detected from the code alone and become apparent only after execution. We formalize this problem as render-feedback-aware constrained code generation: given a natural language specification, the model must generate executable code whose rendered output satisfies structured quality criteria that can be evaluated only after rendering. To address this problem, we introduce OmniManim, a render-feedback-aware educational animation generation framework built around a shared scene state, explicit visual planning, structured post-render diagnostics, and localized repair. Within OmniManim, the Vision Agent is a task-specific visual planning module: it predicts sparse keyframe layouts with coarse-to-fine bounding-box denoising and optimizes an interpolation-aware objective to reduce intermediate-frame failures induced by downstream animation interpolation. We further construct two datasets, ManimLayout-1K and EduRequire-500, and provide a reproducible evaluation protocol covering executability, instructional quality, visual quality, and efficiency. On EduRequire-500, OmniManim improves measured render quality over both single-model baselines and existing multi-agent frameworks. Systematic ablation studies further verify that explicit visual planning, especially its coarse spatial prior, bounding-box refinement, and interpolation-aware optimization, is central to these gains.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 38 of its 42 references that are in the map. First posted on arXiv 15 May 2026.
Builds on
38 of 42 references.
References · 38
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2023GPT-4 Technical Reportcited by 18,396
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2020Denoising Diffusion Implicit Modelscited by 9,781
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023Visual Instruction Tuningcited by 7,869
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
26 more
- 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
- 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
- 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
- 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
- 2023Teaching Large Language Models to Self-Debugcited by 988
- 2023ChatDev: Communicative Agents for Software Developmentcited by 919
- 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 492
- 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 455
- 2023LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generationcited by 259
- 2023LayoutDM: Discrete Diffusion Model for Controllable Layout Generationcited by 143
- 2019LayoutGAN: Generating Graphic Layouts with Wireframe Discriminatorscited by 131
- 2020LayoutTransformer: Layout Generation and Completion with Self-attentioncited by 123
- 2019LayoutVAE: Stochastic Scene Layout Generation From a Label Setcited by 119
- 2024MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualizationcited by 102
- 2023LayoutDM: Transformer-based Diffusion Model for Layout Generationcited by 57
- 2023LayoutDiffusion: Improving Graphic Layout Generation by Discrete Diffusion Probabilistic Modelscited by 41
- 2023DiagrammerGPT: Generating Open-Domain, Open-Platform Diagrams via LLM Planningcited by 29
- 2025Code2Video: A Code-centric Paradigm for Educational Video Generationcited by 27
- 2024Towards Aligned Layout Generation via Diffusion Model with Aesthetic Constraintscited by 26
- 2025From EduVisBench to EduVisAgent: A Benchmark and Multi-Agent Framework for Reasoning-Driven Pedagogical Visualizationcited by 5
- 2026ManiBench: A Benchmark for Testing Visual-Logic Drift and Syntactic Hallucinations in Manim Code Generationcited by 2
Built on by
Not cited yet.