# Speech-Synchronized Whiteboard Generation via VLM-Driven Structured Drawing Representations

arXiv: https://arxiv.org/abs/2603.25870 · HTML: https://mlsanity.com/paper/2603.25870/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 10 references that are in the map.
- First posted on arXiv 26 Mar 2026.

## Builds on (most cited first, 6 of 6)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Factuality Matters: When Image Generation and Editing Meet Structured Visuals](https://mlsanity.com/paper/2510.05091/) (2025)
- [Data Player: Automatic Generation of Data Videos with Narration-Animation Interplay](https://mlsanity.com/paper/2308.04703/) (2023)
- [WonderFlow: Narration-Centric Design of Animated Data Videos](https://mlsanity.com/paper/2308.04040/) (2023)
- [Generating Narrated Lecture Videos from Slides with Synchronized Highlights](https://mlsanity.com/paper/2505.02966/) (2025)

## Built on by (most cited first, 1 of 1)

- [VoCa: Designing Speech-Canvas Interaction for Voice-Based Conversational Agents](https://mlsanity.com/paper/2610.04706/) (2026)

## Abstract (arXiv)

Creating whiteboard-style educational videos demands precise coordination between freehand illustrations and spoken narration, yet no existing method addresses this multimodal synchronization problem with structured, reproducible drawing representations. We present the first dataset of 24 paired Excalidraw demonstrations with narrated audio, where every drawing element carries millisecond-precision creation timestamps spanning 8 STEM domains. Using this data, we study whether a vision-language model (Qwen2-VL-7B), fine-tuned via LoRA, can predict full stroke sequences synchronized to speech from only 24 demonstrations. Our topic-stratified five-fold evaluation reveals that timestamp conditioning significantly improves temporal alignment over ablated baselines, while the model generalizes across unseen STEM topics. We discuss transferability to real classroom settings and release our dataset and code to support future research in automated educational content generation.