mlsanityData to 14 Oct 2026

DiffusionCinema: Text-to-Aerial Cinematography

arXiv 2601.17412 ↗First posted 24 Jan 2026cs.ROcited by 0

On the map: Diffusion models → Diffusion and flow policies for robotics

Abstract (arXiv)

We propose a novel Unmanned Aerial Vehicles (UAV) assisted creative capture system that leverages diffusion models to interpret high-level natural language prompts and automatically generate optimal flight trajectories for cinematic video recording. Instead of manually piloting the drone, the user simply describes the desired shot (e.g., "orbit around me slowly from the right and reveal the background waterfall"). Our system encodes the prompt along with an initial visual snapshot from the onboard camera, and a diffusion model samples plausible spatio-temporal motion plans that satisfy both the scene geometry and shot semantics. The generated flight trajectory is then executed autonomously by the UAV to record smooth, repeatable video clips that match the prompt. User evaluation using NASA-TLX showed a significantly lower overall workload with our interface (M = 21.6) compared to a traditional remote controller (M = 58.1), demonstrating a substantial reduction in perceived effort. Mental demand (M = 11.5 vs. 60.5) and frustration (M = 14.0 vs. 54.5) were also markedly lower for our system, confirming clear usability advantages in autonomous text-driven flight control. This project demonstrates a new interaction paradigm: text-to-cinema flight, where diffusion models act as the "creative operator" converting story intentions directly into aerial motion.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 12 of its 29 references that are in the map. First posted on arXiv 24 Jan 2026.

Builds on

12 of 29 references.

References · 12

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  2. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  3. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  4. 2020ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAMcited by 878
  5. 2023A Survey on Video Diffusion Modelscited by 206
  6. 2023AerialVLN: Vision-and-Language Navigation for UAVscited by 121
  7. 2024Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodologycited by 95
  8. 2025UAV-VLA: Vision-Language-Action System for Large Scale Aerial Mission Generationcited by 28
  9. 2025RaceVLA: VLA-based Racing Drone Navigation with Human-like Behaviourcited by 23
  10. 2025CognitiveDrone: A VLA Model and Evaluation Benchmark for Real-Time Cognitive Task Solving and Reasoning in UAVscited by 22
  11. 2021Human-Piloted Drone Racing: Visual Processing and Controlcited by 9
  12. 2025EmbodiedDiffusion: End-to-End Traversability-Guided Visual Diffusion for Heterogeneous Robot Navigationcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Agentic Aerial Cinematography: From Dialogue Cues to Cinematic Trajectories
  2. 2026Aero-World: Action-Conditioned Aerial Video Generation from Inertial Controls
  3. 2025FlightDiffusion: Revolutionising Autonomous Drone Training with Diffusion Models Generating FPV Video
  4. 2025UAV-Flow Colosseo: A Real-World Benchmark for Flying-on-a-Word UAV Imitation Learning
  5. 2026AeroAct: Action-Centered World-Action Models for Language-Conditioned Quadrotor Flight
  6. 2023Aerial Diffusion: Text Guided Ground-to-Aerial View Translation from a Single Image using Diffusion Models
  7. 2026DeTrack: A Benchmark and Altitude-Aware Dual World Model for Drone-embodied Tracking
  8. 2023Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfer