# ReelFramer: Human-AI Co-Creation for News-to-Video Translation

arXiv: https://arxiv.org/abs/2304.09653 · HTML: https://mlsanity.com/paper/2304.09653/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 87 references that are in the map.
- First posted on arXiv 19 Apr 2023.

## Builds on (most cited first, 26 of 26)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [COMET: Commonsense Transformers for Automatic Knowledge Graph Construction](https://mlsanity.com/paper/1906.05317/) (2019)
- [AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts](https://mlsanity.com/paper/2110.01691/) (2021)
- [Plan-And-Write: Towards Better Automatic Storytelling](https://mlsanity.com/paper/1811.05701/) (2018)
- [DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Models](https://mlsanity.com/paper/2202.04053/) (2022)
- [Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionals](https://mlsanity.com/paper/2209.14958/) (2022)
- [PromptChainer: Chaining Large Language Model Prompts through Visual Programming](https://mlsanity.com/paper/2203.06566/) (2022)
- [Sparks: Inspiration for Science Writing using Language Models](https://mlsanity.com/paper/2110.07640/) (2021)
- [StoryDALL-E: Adapting Pretrained Text-to-Image Transformers for Story Continuation](https://mlsanity.com/paper/2209.06192/) (2022)
- [Narrative Planning: Balancing Plot and Character](https://mlsanity.com/paper/1401.3841/) (2014)
- [The Next Chapter: A Study of Large Language Models in Storytelling](https://mlsanity.com/paper/2301.09790/) (2023)
- [VISAR: A Human-AI Argumentative Writing Assistant with Visual Programming and Rapid Draft Prototyping](https://mlsanity.com/paper/2304.07810/) (2023)
- [Automated Storytelling via Causal, Commonsense Plot Ordering](https://mlsanity.com/paper/2009.00829/) (2020)
- [Opal: Multimodal Image Generation for News Illustration](https://mlsanity.com/paper/2204.09007/) (2022)
- [PopBlends: Strategies for Conceptual Blending with Large Language Models](https://mlsanity.com/paper/2111.04920/) (2021)
- [Open-world Story Generation with Structured Knowledge Enhancement: A Comprehensive Survey](https://mlsanity.com/paper/2212.04634/) (2022)
- [An Interaction Framework for Studying Co-Creative AI](https://mlsanity.com/paper/1903.09709/) (2019)
- [Frustratingly Short Attention Spans in Neural Language Modeling](https://mlsanity.com/paper/1702.04521/) (2017)
- [CodeToon: Story Ideation, Auto Comic Generation, and Structure Mapping for Code-Driven Storytelling](https://mlsanity.com/paper/2208.12981/) (2022)
- [Can ChatGPT-like Generative Models Guarantee Factual Accuracy? On the Mistakes of New Generation Search Engines](https://mlsanity.com/paper/2304.11076/) (2023)
- [Mechanical Novel: Crowdsourcing Complex Work through Reflection and Revision](https://mlsanity.com/paper/1611.02682/) (2016)
- [Neural Storyboard Artist: Visualizing Stories with Coherent Image Sequences](https://mlsanity.com/paper/1911.10460/) (2019)

## Built on by (most cited first, 19 of 19)

- [Challenges and Applications of Large Language Models](https://mlsanity.com/paper/2307.10169/) (2023)
- [Advancements in Generative AI: A Comprehensive Review of GANs, GPT, Autoencoders, Diffusion Model, and Transformers](https://mlsanity.com/paper/2311.10242/) (2023)
- [Not Just Novelty: A Longitudinal Study on Utility and Customization of an AI Workflow](https://mlsanity.com/paper/2402.09894/) (2024)
- [Which Contributions Deserve Credit? Perceptions of Attribution in Human-AI Co-Creation](https://mlsanity.com/paper/2502.18357/) (2025)
- ["Ownership, Not Just Happy Talk": Co-Designing a Participatory Large Language Model for Journalism](https://mlsanity.com/paper/2501.17299/) (2025)
- [LlamaLens: Specialized Multilingual LLM for Analyzing News and Social Media Content](https://mlsanity.com/paper/2410.15308/) (2024)
- [Lotus: Creating Short Videos From Long Videos With Abstractive and Extractive Summarization](https://mlsanity.com/paper/2502.07096/) (2025)
- [AutoCut: End-to-end advertisement video editing based on multimodal discretization and controllable generation](https://mlsanity.com/paper/2603.28366/) (2026)
- [Exploring the Needs of Practising Musicians in Co-Creative AI Through Co-Design](https://mlsanity.com/paper/2502.09055/) (2025)
- [AI Rivalry as a Craft: How Resisting and Embracing Generative AI Reshape Writing Professions](https://mlsanity.com/paper/2503.09901/) (2025)
- [FeedQUAC: Quick Unobtrusive AI-Generated Commentary](https://mlsanity.com/paper/2504.16416/) (2025)
- [Vidmento: Creating Video Stories Through Context-Aware Expansion With Generative Video](https://mlsanity.com/paper/2601.22013/) (2026)
- [Dynamic and Super-Personalized Media Ecosystem Driven by Generative AI: Unpredictable Plays Never Repeating The Same](https://mlsanity.com/paper/2402.12412/) (2024)
- [Adaptive Human-Agent Teaming: A Review of Empirical Studies from the Process Dynamics Perspective](https://mlsanity.com/paper/2504.10918/) (2025)
- [Facilitating Longitudinal Interaction Studies of AI Systems](https://mlsanity.com/paper/2508.10252/) (2025)
- [AnimAgents: Coordinating Multi-Stage Animation Pre-Production with Human-Multi-Agent Collaboration](https://mlsanity.com/paper/2511.17906/) (2025)
- [Beyond Input-Output: Rethinking Creativity through Design-by-Analogy in Human-AI Collaboration](https://mlsanity.com/paper/2602.09423/) (2026)
- [Human-AI Schema Discovery and Application for Creative Problem Solving](https://mlsanity.com/paper/2508.05045/) (2025)
- [PaperTok: Exploring the Use of Generative AI for Creating Short-form Videos for Research Communication](https://mlsanity.com/paper/2601.18218/) (2026)

## Abstract (arXiv)

Short videos on social media are the dominant way young people consume content. News outlets aim to reach audiences through news reels -- short videos conveying news -- but struggle to translate traditional journalistic formats into short, entertaining videos. To translate news into social media reels, we support journalists in reframing the narrative. In literature, narrative framing is a high-level structure that shapes the overall presentation of a story. We identified three narrative framings for reels that adapt social media norms but preserve news value, each with a different balance of information and entertainment. We introduce ReelFramer, a human-AI co-creative system that helps journalists translate print articles into scripts and storyboards. ReelFramer supports exploring multiple narrative framings to find one appropriate to the story. AI suggests foundational narrative details, including characters, plot, setting, and key information. ReelFramer also supports visual framing; AI suggests character and visual detail designs before generating a full storyboard. Our studies show that narrative framing introduces the necessary diversity to translate various articles into reels, and establishing foundational details helps generate scripts that are more relevant and coherent. We also discuss the benefits of using narrative framing and foundational details in content retargeting.