mlsanityData to 7 Oct 2026

Text Prompting for Multi-Concept Video Customization by Autoregressive Generation

arXiv 2405.13951 ↗First posted 22 May 2024cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

We present a method for multi-concept customization of pretrained text-to-video (T2V) models. Intuitively, the multi-concept customized video can be derived from the (non-linear) intersection of the video manifolds of the individual concepts, which is not straightforward to find. We hypothesize that sequential and controlled walking towards the intersection of the video manifolds, directed by text prompting, leads to the solution. To do so, we generate the various concepts and their corresponding interactions, sequentially, in an autoregressive manner. Our method can generate videos of multiple custom concepts (subjects, action and background) such as a teddy bear running towards a brown teapot, a dog playing violin and a teddy bear swimming in the ocean. We quantitatively evaluate our method using videoCLIP and DINO scores, in addition to human evaluation. Videos for results presented in this paper can be found at https://github.com/divyakraman/MultiConceptVideo2024.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 41 of its 43 references that are in the map. First posted on arXiv 22 May 2024.

Builds on

41 of 43 references.

References · 41

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  3. 2022Classifier-Free Diffusion Guidancecited by 5,717
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  5. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  6. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  7. 2022Video Diffusion Modelscited by 2,280
  8. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  9. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  10. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  11. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
  12. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
29 more
  1. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
  2. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  3. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  4. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  5. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 696
  6. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 619
  7. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 589
  8. 2017The Perception-Distortion Tradeoffcited by 582
  9. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  10. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 462
  11. 2023SVDiff: Compact Parameter Space for Diffusion Fine-Tuningcited by 338
  12. 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generationcited by 330
  13. 2023Pix2Video: Video Editing using Image Diffusioncited by 318
  14. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attentioncited by 313
  15. 2023Video-P2P: Video Editing with Cross-attention Controlcited by 312
  16. 2022Flexible Diffusion Modeling of Long Videoscited by 294
  17. 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Modelscited by 284
  18. 2023Subject-Diffusion:Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuningcited by 191
  19. 2023Dreamix: Video Diffusion Models are General Video Editorscited by 189
  20. 2023Video Probabilistic Diffusion Models in Projected Latent Spacecited by 171
  21. 2023NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generationcited by 170
  22. 2023Zero-Shot Video Editing Using Off-The-Shelf Image Diffusion Modelscited by 135
  23. 2023VideoControlNet: A Motion-Guided Video-to-Video Translation Framework by Using Diffusion Model with ControlNetcited by 83
  24. 2023CCEdit: Creative and Controllable Video Editing via Diffusion Modelscited by 80
  25. 2022Visual Prompt Tuning for Generative Transfer Learningcited by 75
  26. 2023Free-Bloom: Zero-Shot Text-to-Video Generator with LLM Director and LDM Animatorcited by 74
  27. 2023Towards Consistent Video Editing with Text-to-Image Diffusion Modelscited by 33
  28. 2023InFusion: Inject and Attention Fusion for Multi Concept Zero-Shot Text-based Video Editingcited by 17
  29. 2023ConditionVideo: Training-Free Condition-Guided Text-to-Video Generationcited by 5

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 2.

3 papers.

  1. 2025Movie Weaver: Tuning-Free Multi-Concept Video Personalization with Anchored Promptscited by 28
  2. 2024ImPoster: Text and Frequency Guidance for Subject Driven Action Personalization using Diffusion Modelscited by 2
  3. 2025Zero-Shot Personalized Camera Motion Control for Image-to-Video Synthesiscited by 0