mlsanityData to 7 Oct 2026

Text-Driven Stylization of Video Objects

arXiv 2206.12396 ↗First posted 24 Jun 2022cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

We tackle the task of stylizing video objects in an intuitive and semantic manner following a user-specified text prompt. This is a challenging task as the resulting video must satisfy multiple properties: (1) it has to be temporally consistent and avoid jittering or similar artifacts, (2) the resulting stylization must preserve both the global semantics of the object and its fine-grained details, and (3) it must adhere to the user-specified text prompt. To this end, our method stylizes an object in a video according to two target texts. The first target text prompt describes the global semantics and the second target text prompt describes the local semantics. To modify the style of an object, we harness the representational power of CLIP to get a similarity score between (1) the local target text and a set of local stylized views, and (2) a global target text and a set of stylized global views. We use a pretrained atlas decomposition network to propagate the edits in a temporally consistent manner. We demonstrate that our method can generate consistent style changes over time for a variety of objects and videos, that adhere to the specification of the target texts. We also show how varying the specificity of the target texts and augmenting the texts with a set of prefixes results in stylizations with different levels of detail. Full results are given on our project webpage: https://sloeschcke.github.io/Text-Driven-Stylization-of-Video-Objects/

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 30 of its 32 references that are in the map. First posted on arXiv 24 Jun 2022.

Builds on

30 of 32 references.

References · 30

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  6. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  8. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  9. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  10. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  11. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  12. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
18 more
  1. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  2. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
  3. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 332
  4. 2022Text2LIVE: Text-Driven Layered Image and Video Editingcited by 312
  5. 2022VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidancecited by 285
  6. 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
  7. 2020Space-Time Correspondence as a Contrastive Random Walkcited by 239
  8. 2021CLIPstyler: Image Style Transfer with a Single Text Conditioncited by 203
  9. 2021CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoderscited by 186
  10. 2021StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image Generatorscited by 164
  11. 2021Layered Neural Atlases for Consistent Video Editingcited by 137
  12. 2021Paint by Wordcited by 107
  13. 2022Deformable Sprites for Unsupervised Video Decompositioncited by 61
  14. 20213DStyleNet: Creating 3D Shapes with Geometric and Texture Style Variationscited by 58
  15. 2016Video Propagation Networkscited by 45
  16. 2020Interactive Video Stylization Using Few-Shot Patch-Based Trainingcited by 43
  17. 2021ClipMatrix: Text-controlled Creation of 3D Textured Meshescited by 40
  18. 2021Image-Based CLIP-Guided Essence Transfercited by 39

Built on by

Cited by year

Citing papers per year: 2023: 9; 2024: 2; 2025: 1.

12 papers.

  1. 2023Pix2Video: Video Editing using Image Diffusioncited by 318
  2. 2023Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfercited by 124
  3. 2024Video as the New Language for Real-World Decision Makingcited by 86
  4. 2023Edit-A-Video: Single Video Editing with Object-Aware Consistencycited by 65
  5. 2023AI-Generated Content (AIGC): A Surveycited by 59
  6. 2023Shape-aware Text-driven Layered Video Editingcited by 54
  7. 2023VidEdit: Zero-Shot and Spatially Aware Text-Driven Video Editingcited by 36
  8. 2023Blind Video Deflickering by Neural Filtering with a Flawed Atlascited by 27
  9. 2023Edit Temporal-Consistent Videos with Image Diffusion Modelcited by 11
  10. 2025EverybodyDance: Bipartite Graph-Based Identity Correspondence for Multi-Character Animationcited by 3
  11. 2024Socialized Learning: A Survey of the Paradigm Shift for Edge Intelligence in Networked Systemscited by 2
  12. 2023FreeFlow: A Comprehensive Understanding on Diffusion Probabilistic Models via Optimal Transportcited by 0