mlsanityData to 7 Oct 2026

Training and Agentic Inference Strategies for LLM-based Manim Animation Generation

arXiv 2604.18364 ↗First posted 20 Apr 2026cs.AI cs.GR cs.MAcited by 2

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Generating programmatic animation using libraries such as Manim presents unique challenges for Large Language Models (LLMs), requiring spatial reasoning, temporal sequencing, and familiarity with domain-specific APIs that are underrepresented in general pre-training data. A systematic study of how training and inference strategies interact in this setting is lacking in current research. This study introduces ManimTrainer, a training pipeline that combines Supervised Fine-tuning (SFT) with Reinforcement Learning (RL) based Group Relative Policy Optimisation (GRPO) using a unified reward signal that fuses code and visual assessment signals, and ManimAgent, an inference pipeline featuring Renderer-in-the-loop (RITL) and API documentation-augmented RITL (RITL-DOC) strategies. Using these techniques, this study presents the first unified training and inference study for text-to-code-to-video transformation with Manim. It evaluates 17 open-source sub-30B LLMs across nine combinations of training and inference strategies using ManimBench. Results show that SFT generally improves code quality, while GRPO enhances visual outputs and increases the models' responsiveness to extrinsic signals during self-correction at inference time. The Qwen 3 Coder 30B model with GRPO and RITL-DOC achieved the highest overall performance, with a 94% Render Success Rate (RSR) and 85.7% Visual Similarity (VS) to reference videos, surpassing the baseline GPT-4.1 model by +3 percentage points in VS. Additionally, the analysis shows that the correlation between code and visual metrics strengthens with SFT and GRPO but weakens with inference-time enhancements, highlighting the complementary roles of training and agentic inference strategies in Manim animation generation.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 30 of its 37 references that are in the map. First posted on arXiv 20 Apr 2026.

Builds on

30 of 37 references.

References · 30

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  8. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  9. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  10. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  11. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  12. 2022Training Compute-Optimal Large Language Modelscited by 2,927
18 more
  1. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
  2. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  3. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  4. 2023Teaching Large Language Models to Self-Debugcited by 963
  5. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
  6. 2020CodeBLEU: a Method for Automatic Evaluation of Code Synthesiscited by 468
  7. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 342
  8. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 230
  9. 2023InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedbackcited by 229
  10. 2024CodeRAG-Bench: Can Retrieval Augment Code Generation?cited by 89
  11. 2025Code2Video: A Code-centric Paradigm for Educational Video Generationcited by 23
  12. 2025TheoremExplainAgent: Towards Video-based Multimodal Explanations for LLM Theorem Understandingcited by 20
  13. 2025VinciCoder: Unifying Multimodal Code Generation via Coarse-to-fine Visual Reinforcement Learningcited by 18
  14. 2025ReLook: Vision-Grounded RL with a Multimodal LLM Critic for Agentic Web Codingcited by 12
  15. 2025Can LLMs Correct Themselves? A Benchmark of Self-Correction in LLMscited by 12
  16. 2024LogoMotion: Visually-Grounded Code Synthesis for Creating and Editing Animationcited by 10
  17. 2025JanusCoder: Towards a Foundational Visual-Programmatic Interface for Code Intelligencecited by 7
  18. 2025Manimator: Transforming Research Papers into Visual Explanationscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Training and Benchmarking Code Generation for Physics-Inspired Animations
  2. 2026SFT or RL for Tool-Calling Agents? A Controlled Study Across Data, Method, and Scale
  3. 2025TempSamp-R1: Effective Temporal Sampling with Reinforcement Fine-Tuning for Video LLMs
  4. 2025Training-Free Group Relative Policy Optimization
  5. 2025Breaking the SFT Plateau: Multimodal Structured Reinforcement Learning for Chart-to-Code Generation
  6. 2026Dr. Seg: Revisiting GRPO Training for Visual Large Language Models through Perception-Oriented Design
  7. 2025AdaTooler-V: Adaptive Tool-Use for Images and Videos
  8. 2024Self-Training Large Language Models for Improved Visual Program Synthesis With Visual Reinforcement