mlsanityData to 14 Oct 2026

MakeAnything: Harnessing Diffusion Transformers for Multi-Domain Procedural Sequence Generation

arXiv 2502.01572 ↗First posted 3 Feb 2025cs.CVcited by 45

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

A hallmark of human intelligence is the ability to create complex artifacts through structured multi-step processes. Generating procedural tutorials with AI is a longstanding but challenging goal, facing three key obstacles: (1) scarcity of multi-task procedural datasets, (2) maintaining logical continuity and visual consistency between steps, and (3) generalizing across multiple domains. To address these challenges, we propose a multi-domain dataset covering 21 tasks with over 24,000 procedural sequences. Building upon this foundation, we introduce MakeAnything, a framework based on the diffusion transformer (DIT), which leverages fine-tuning to activate the in-context capabilities of DIT for generating consistent procedural sequences. We introduce asymmetric low-rank adaptation (LoRA) for image generation, which balances generalization capabilities and task-specific performance by freezing encoder parameters while adaptively tuning decoder layers. Additionally, our ReCraft model enables image-to-process generation through spatiotemporal consistency constraints, allowing static images to be decomposed into plausible creation sequences. Extensive experiments demonstrate that MakeAnything surpasses existing methods, setting new performance benchmarks for procedural generation tasks.

Key facts. Cited by 45 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 19 of its 45 references that are in the map. First posted on arXiv 3 Feb 2025.

Builds on

19 of 45 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024DiffSim: Taming Diffusion Models for Evaluating Visual Similarity19 Dec
  2. 2024Stable-Makeup: When Real-World Makeup Transfer Meets Diffusion Model12 Mar
  3. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models13 Aug
  4. 2023Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Models25 May
  5. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models16 Feb
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb

Other references · 13

  1. 2020Denoising Diffusion Implicit Modelscited by 9,781
  2. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  3. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  4. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
  5. 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 340
  6. 2017A Neural Representation of Sketch Drawingscited by 313
  7. 2024Asymmetry in Low-Rank Adapters of Foundation Modelscited by 86
  8. 2024ProcessPainter: Learn Painting Process from Sequence Datacited by 38
  9. 2024Stable-Hair: Real-World Hair Transfer via Diffusion Modelcited by 35
  10. 2020Multi-modal Attention for Speech Emotion Recognitioncited by 29
  11. 2019Neural Painters: A learned differentiable constraint for generating brushstroke paintingscited by 28
  12. 2018Learning to Sketch with Deep Q Networks and Demonstrated Strokescited by 6
1 more
  1. 2024Grid: Omni Visual Generationcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 29; 2026: 16.

45 papers.

Sort:

By area:

  1. 2025EasyControl: Adding Efficient and Flexible Control for Diffusion Transformercited by 119
  2. 2025Controllable Video Generation: A Surveycited by 98
  3. 2025EEdit: Rethinking the Spatial and Temporal Redundancy for Efficient Image Editingcited by 45
  4. 2025OmniConsistency: Learning Style-Agnostic Consistency from Paired Stylization Datacited by 32
  5. 2025EasyText: Controllable Diffusion Transformer for Multilingual Text Renderingcited by 26
  6. 2025RelationAdapter: Learning and Transferring Visual Relation with Diffusion Transformerscited by 26
  7. 2025PhotoDoodle: Learning Artistic Image Editing from Few-Shot Pairwise Datacited by 24
  8. 2025All Patches Matter, More Patches Better: Enhance AI-Generated Image Detection via Panoptic Patch Learningcited by 24
  9. 2026FastVMT: Eliminating Redundancy in Video Motion Transfercited by 23
  10. 2025DiffDecompose: Layer-Wise Decomposition of Alpha-Composited Images via Diffusion Transformerscited by 20
  11. 2025GRE Suite: Geo-localization Inference via Fine-Tuned Vision-Language Models and Enhanced Reasoning Chainscited by 14
  12. 2025Mitty: Diffusion-based Human-to-Robot Video Generationcited by 14
33 more
  1. 2025OmniPSD: Layered PSD Generation with Diffusion Transformercited by 13
  2. 2026Group Editing: Edit Multiple Images in One Gocited by 11
  3. 2025WorldWander: Bridging Egocentric and Exocentric Worlds in Video Generationcited by 9
  4. 2025Autoregressive Images Watermarking through Lexical Biasing: An Approach Resistant to Regeneration Attackcited by 8
  5. 2026VISTA: Triplet-Supervised Video Style Transfer with Diffusion Transformerscited by 8
  6. 2026PAI-Studio: Cinematic Video Background Replacement with Camera-Aware Motioncited by 8
  7. 2025FeRA: Frequency-Energy Constrained Routing for Effective Diffusion Adaptation Fine-Tuningcited by 7
  8. 2025PaCo-RL: Advancing Reinforcement Learning for Consistent Image Generation with Pairwise Reward Modelingcited by 7
  9. 2025Edit2Perceive: Image Editing Diffusion Models Are Strong Dense Perceiverscited by 5
  10. 2025OmniRefiner: Reinforcement-Guided Local Diffusion Refinementcited by 5
  11. 2025The Consistency Critic: Correcting Inconsistencies in Generated Images via Reference-Guided Attentive Alignmentcited by 5
  12. 2025CineVerse: Consistent Keyframe Synthesis for Cinematic Scene Compositioncited by 4
  13. 2025NumeriKontrol: Adding Numeric Control to Diffusion Transformers for Instruction-based Image Editingcited by 4
  14. 2025IC-Effect: Precise and Efficient Video Effects Editing via In-Context Learningcited by 4
  15. 2025Loom: Diffusion-Transformer for Interleaved Generationcited by 4
  16. 2026OmniHumanoid: Streaming Cross-Embodiment Video Generation with Paired-Free Adaptationcited by 4
  17. 2025TanDiT: Tangent-Plane Diffusion Transformer for High-Quality 360deg Panorama Generationcited by 3
  18. 2025Chain-of-Cooking:Cooking Process Visualization via Bidirectional Chain-of-Thought Guidancecited by 3
  19. 2025MultiMotion: Multi Subject Video Motion Transfer via Video Diffusion Transformercited by 3
  20. 2026Image-to-Video Diffusion: From Foundations to Open Frontierscited by 3
  21. 2026TryOnCrafter: Unleashing Camera Trajectories for Realistic Video Virtual Try-on via a Renderable 4D Try-on Proxycited by 3
  22. 2025Markovian Scale Prediction: A New Era of Visual Autoregressive Generationcited by 2
  23. 2026SIGMA: Selective-Interleaved Generation with Multi-Attribute Tokenscited by 2
  24. 2026EasyVFX: Frequency-Driven Decoupling for Resource-Efficient VFX Generationcited by 2
  25. 2025Image Generation as a Visual Planner for Robotic Manipulationcited by 1
  26. 2025TokenPure: Watermark Removal through Tokenized Appearance and Structural Guidancecited by 1
  27. 2026InstanceAnimator: Multi-Instance Sketch Video Colorizationcited by 1
  28. 2026EditTransfer++: Toward Faithful and Efficient Visual-Prompt-Guided Image Editingcited by 1
  29. 2026SWEET: Sparse World Modeling with Image Editing for Embodied Task Executioncited by 1
  30. 2026Uni-Animator: Towards Unified Visual Colorizationcited by 0
  31. 2026InstructionCrafter: Generating Consistent and High-Fidelity Visual Instructionscited by 0
  32. 2026PAI-Actor: Cinematic Multi-Character Replacement in Dynamic Scenescited by 0
  33. 2026VectorHarness: Recovering Editable, Relation-Preserving Structure from Scientific Graphicscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Generating Coherent Sequences of Visual Illustrations for Real-World Manual Tasks
  2. 2025CookingDiffusion: Cooking Procedural Image Generation with Stable Diffusion
  3. 2026Reinforcing the Generation Order of Multimodal Masked Diffusion Models
  4. 2023CycleNet: Rethinking Cycle Consistency in Text-Guided Diffusion for Image Manipulation
  5. 2026WorldGuide: Goal-Directed Video World Model for Procedural Task Execution
  6. 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs
  7. 2025In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformer
  8. 2026Think in Strokes, Not Pixels: Process-Driven Image Generation via Interleaved Reasoning