mlsanityData to 14 Oct 2026

See Before You Code: Learning Visual Priors for Spatially Aware Educational Animation Generation

arXiv 2605.15585 ↗First posted 15 May 2026cs.AI cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models can generate executable code for educational animations, but the resulting renders often exhibit visual defects, including element overlap, misalignment, and broken animation continuity. These defects cannot be reliably detected from the code alone and become apparent only after execution. We formalize this problem as render-feedback-aware constrained code generation: given a natural language specification, the model must generate executable code whose rendered output satisfies structured quality criteria that can be evaluated only after rendering. To address this problem, we introduce OmniManim, a render-feedback-aware educational animation generation framework built around a shared scene state, explicit visual planning, structured post-render diagnostics, and localized repair. Within OmniManim, the Vision Agent is a task-specific visual planning module: it predicts sparse keyframe layouts with coarse-to-fine bounding-box denoising and optimizes an interpolation-aware objective to reduce intermediate-frame failures induced by downstream animation interpolation. We further construct two datasets, ManimLayout-1K and EduRequire-500, and provide a reproducible evaluation protocol covering executability, instructional quality, visual quality, and efficiency. On EduRequire-500, OmniManim improves measured render quality over both single-model baselines and existing multi-agent frameworks. Systematic ablation studies further verify that explicit visual planning, especially its coarse spatial prior, bounding-box refinement, and interpolation-aware optimization, is central to these gains.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 38 of its 42 references that are in the map. First posted on arXiv 15 May 2026.

Builds on

38 of 42 references.

References · 38

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2020Denoising Diffusion Implicit Modelscited by 9,781
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  6. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  8. 2023Visual Instruction Tuningcited by 7,869
  9. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  10. 2022Flow Matching for Generative Modelingcited by 5,702
  11. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  12. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
26 more
  1. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  2. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  3. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  4. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  5. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  6. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
  7. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  8. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  9. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  10. 2023Teaching Large Language Models to Self-Debugcited by 988
  11. 2023ChatDev: Communicative Agents for Software Developmentcited by 919
  12. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 492
  13. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 455
  14. 2023LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generationcited by 259
  15. 2023LayoutDM: Discrete Diffusion Model for Controllable Layout Generationcited by 143
  16. 2019LayoutGAN: Generating Graphic Layouts with Wireframe Discriminatorscited by 131
  17. 2020LayoutTransformer: Layout Generation and Completion with Self-attentioncited by 123
  18. 2019LayoutVAE: Stochastic Scene Layout Generation From a Label Setcited by 119
  19. 2024MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualizationcited by 102
  20. 2023LayoutDM: Transformer-based Diffusion Model for Layout Generationcited by 57
  21. 2023LayoutDiffusion: Improving Graphic Layout Generation by Discrete Diffusion Probabilistic Modelscited by 41
  22. 2023DiagrammerGPT: Generating Open-Domain, Open-Platform Diagrams via LLM Planningcited by 29
  23. 2025Code2Video: A Code-centric Paradigm for Educational Video Generationcited by 27
  24. 2024Towards Aligned Layout Generation via Diffusion Model with Aesthetic Constraintscited by 26
  25. 2025From EduVisBench to EduVisAgent: A Benchmark and Multi-Agent Framework for Reasoning-Driven Pedagogical Visualizationcited by 5
  26. 2026ManiBench: A Benchmark for Testing Visual-Logic Drift and Syntactic Hallucinations in Manim Code Generationcited by 2

Built on by

Not cited yet.