# LEARN: A Story-Driven Layout-to-Image Generation Framework for STEM Instruction

arXiv: https://arxiv.org/abs/2508.11153 · HTML: https://mlsanity.com/paper/2508.11153/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 25 references that are in the map.
- First posted on arXiv 15 Aug 2025.

## Builds on (most cited first, 15 of 15)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [GLIGEN: Open-Set Grounded Text-to-Image Generation](https://mlsanity.com/paper/2301.07093/) (2023)
- [Image Segmentation Using Text and Image Prompts](https://mlsanity.com/paper/2112.10003/) (2021)
- [LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generation](https://mlsanity.com/paper/2303.17189/) (2023)
- [Object-driven Text-to-Image Synthesis via Adversarial Training](https://mlsanity.com/paper/1902.10740/) (2019)
- [Inferring Semantic Layout for Hierarchical Text-to-Image Synthesis](https://mlsanity.com/paper/1801.05091/) (2018)
- [CM-GANs: Cross-modal Generative Adversarial Networks for Common Representation Learning](https://mlsanity.com/paper/1710.05106/) (2017)
- [DreamArtist++: Controllable One-Shot Text-to-Image Generation via Positive-Negative Adapter](https://mlsanity.com/paper/2211.11337/) (2022)
- [Zero-Painter: Training-Free Layout Control for Text-to-Image Synthesis](https://mlsanity.com/paper/2406.04032/) (2024)
- [Rethinking The Training And Evaluation of Rich-Context Layout-to-Image Generation](https://mlsanity.com/paper/2409.04847/) (2024)

## Abstract (arXiv)

LEARN is a layout-aware diffusion framework designed to generate pedagogically aligned illustrations for STEM education. It leverages a curated BookCover dataset that provides narrative layouts and structured visual cues, enabling the model to depict abstract and sequential scientific concepts with strong semantic alignment. Through layout-conditioned generation, contrastive visual-semantic training, and prompt modulation, LEARN produces coherent visual sequences that support mid-to-high-level reasoning in line with Bloom's taxonomy while reducing extraneous cognitive load as emphasized by Cognitive Load Theory. By fostering spatially organized and story-driven narratives, the framework counters fragmented attention often induced by short-form media and promotes sustained conceptual focus. Beyond static diagrams, LEARN demonstrates potential for integration with multimodal systems and curriculum-linked knowledge graphs to create adaptive, exploratory educational content. As the first generative approach to unify layout-based storytelling, semantic structure learning, and cognitive scaffolding, LEARN represents a novel direction for generative AI in education. The code and dataset will be released to facilitate future research and practical deployment.