mlsanityData to 7 Oct 2026

CLIP-Actor: Text-Driven Recommendation and Stylization for Animating Human Meshes

arXiv 2206.04382 ↗First posted 9 Jun 2022cs.CV cs.AI cs.GRcited by 41

Not on the map. Why?

Abstract (arXiv)

We propose CLIP-Actor, a text-driven motion recommendation and neural mesh stylization system for human mesh animation. CLIP-Actor animates a 3D human mesh to conform to a text prompt by recommending a motion sequence and optimizing mesh style attributes. We build a text-driven human motion recommendation system by leveraging a large-scale human motion dataset with language labels. Given a natural language prompt, CLIP-Actor suggests a text-conforming human motion in a coarse-to-fine manner. Then, our novel zero-shot neural style optimization detailizes and texturizes the recommended mesh sequence to conform to the prompt in a temporally-consistent and pose-agnostic manner. This is distinctive in that prior work fails to generate plausible results when the pose of an artist-designed mesh does not conform to the text from the beginning. We further propose the spatio-temporal view augmentation and mask-weighted embedding attention, which stabilize the optimization process by leveraging multi-frame human motion and rejecting poorly rendered views. We demonstrate that CLIP-Actor produces plausible and human-recognizable style 3D human mesh in motion with detailed geometry and texture solely from a natural language prompt.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). Builds on 56 of its 69 references that are in the map. First posted on arXiv 9 Jun 2022.

Builds on

56 of 69 references.

References · 56

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  3. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  4. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  5. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
  6. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  7. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  8. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  9. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  10. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
  11. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  12. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
38 more
  1. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  2. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulationcited by 640
  3. 2021RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputscited by 614
  4. 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
  5. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  6. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 497
  7. 2022TEMOS: Generating diverse human motions from textual descriptionscited by 486
  8. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 465
  9. 2021Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesiscited by 464
  10. 2021PointCLIP: Point Cloud Understanding by CLIPcited by 461
  11. 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 438
  12. 2022MotionCLIP: Exposing Human Motion Generation to CLIP Spacecited by 406
  13. 2021CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fieldscited by 369
  14. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 332
  15. 20163D Menagerie: Modeling the 3D shape and pose of animalscited by 310
  16. 2021BABEL: Bodies, Action and Behavior with English Labelscited by 261
  17. 2019Language2Pose: Natural Language Grounded Pose Forecastingcited by 257
  18. 2019Multi-Garment Net: Learning to Dress 3D People from Imagescited by 244
  19. 2021CLIPstyler: Image Style Transfer with a Single Text Conditioncited by 203
  20. 2021CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoderscited by 186
  21. 2021SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networkscited by 181
  22. 2022Embodied Hands: Modeling and Capturing Hands and Bodies Togethercited by 179
  23. 2021Synthesis of Compositional Animations from Textual Descriptionscited by 177
  24. 2020Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wildcited by 152
  25. 2020Reconstructing Hand-Object Interactions in the Wildcited by 143
  26. 2020Combining Implicit Function Learning and Parametric Models for 3D Human Reconstructioncited by 133
  27. 2020Differentiable Rendering: A Surveycited by 123
  28. 2018Robots Learn Social Skills: End-to-End Learning of Co-Speech Gesture Generation for Humanoid Robotscited by 110
  29. 2017Text2Action: Generative Adversarial Synthesis from Language to Actioncited by 106
  30. 2019Three-D Safari: Learning to Estimate Zebra Pose, Shape, and Texture from Images "In the Wild"cited by 106
  31. 2021NPMs: Neural Parametric Models for 3D Deformable Shapescited by 89
  32. 2018Creatures great and SMAL: Recovering the shape and motion of animals from videocited by 84
  33. 2020Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loopcited by 81
  34. 2021MetaAvatar: Learning Animatable Clothed Human Models from Few Depth Imagescited by 78
  35. 2020Neural Deformation Graphs for Globally-consistent Non-rigid Reconstructioncited by 66
  36. 2017Learning a bidirectional mapping between human whole-body motion and natural language using deep recurrent neural networkscited by 61
  37. 2020Neural Non-Rigid Trackingcited by 49
  38. 2021Learning Signal-Agnostic Manifolds of Neural Fieldscited by 45

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 23; 2024: 9; 2025: 4; 2026: 1.

41 papers.

Sort:

By area:

  1. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  2. 2023MotionGPT: Human Motion as a Foreign Languagecited by 521
  3. 2023HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splattingcited by 89
  4. 2023AvatarCraft: Transforming Text into Neural Human Avatars with Parameterized Shape and Pose Controlcited by 84
  5. 2023Paint-it: Text-to-Texture Synthesis via Deep Convolutional Texture Map Optimization and Physically-Based Renderingcited by 84
  6. 2022ClipFace: Text-guided Editing of Textured 3D Morphable Modelscited by 59
  7. 2022Understanding Pure CLIP Guidance for Voxel Grid NeRF Modelscited by 56
  8. 2022Being Comes from Not-being: Open-vocabulary Text-to-Motion Generation with Wordless Trainingcited by 45
  9. 2024HanDiffuser: Text-to-Image Generation With Realistic Hand Appearancescited by 42
  10. 2023Sound to Visual Scene Generation by Audio-to-Visual Latent Alignmentcited by 41
  11. 2023PoseFix: Correcting 3D Human Poses with Natural Languagecited by 38
  12. 2023ShapeGPT: 3D Shape Generation with A Unified Multi-modal Language Modelcited by 38
29 more
  1. 2023AvatarBooth: High-Quality and Customizable 3D Human Avatar Generationcited by 37
  2. 2023ViT-Lens: Towards Omni-modal Representationscited by 29
  3. 2024Controllable Text-to-3D Generation via Surface-Aligned Gaussian Splattingcited by 26
  4. 2023What is the Best Automated Metric for Text to Motion Generation?cited by 19
  5. 2023ZeroAvatar: Zero-shot 3D Avatar Generation from a Single Imagecited by 18
  6. 2023Text2Scene: Text-driven Indoor Scene Stylization with Part-aware Detailscited by 18
  7. 2023XAGen: 3D Expressive Human Avatars Generationcited by 17
  8. 2025A Survey of Foundation Model-Powered Recommender Systems: From Feature-Based, Generative to Agentic Paradigmscited by 17
  9. 2024TexDreamer: Towards Zero-Shot High-Fidelity 3D Human Texture Generationcited by 14
  10. 2022PoseScript: Linking 3D Human Poses and Natural Languagecited by 13
  11. 2024STAR: Skeleton-aware Text-based 4D Avatar Generation with In-Network Motion Retargetingcited by 13
  12. 2023Narrator: Towards Natural Control of Human-Scene Interaction Generation via Relationship Reasoningcited by 10
  13. 2023TextManiA: Enriching Visual Feature by Text-driven Manifold Augmentationcited by 9
  14. 2023MVControl: Adding Conditional Control to Multi-view Diffusion for Controllable Text-to-3D Generationcited by 8
  15. 2023Text-Guided 3D Face Synthesis -- From Generation to Editingcited by 8
  16. 2024PoseEmbroider: Towards a 3D, Visual, Semantic-aware Human Pose Representationcited by 8
  17. 2023Semantify: Simplifying the Control of 3D Morphable Models using CLIPcited by 7
  18. 2024HumanCoser: Layered 3D Human Generation via Semantic-Aware Diffusion Modelcited by 7
  19. 2023ET3D: Efficient Text-to-3D Generation via Multi-View Distillationcited by 6
  20. 2023Joint2Human: High-quality 3D Human Generation via Compact Spherical Embedding of 3D Jointscited by 6
  21. 2023A Large-Scale 3D Face Mesh Video Dataset via Neural Re-parameterized Optimizationcited by 5
  22. 2024DAGSM: Disentangled Avatar Generation with GS-enhanced Meshcited by 5
  23. 2024Sound2Vision: Generating Diverse Visuals from Audio through Cross-Modal Latent Alignmentcited by 5
  24. 2025InfiniHuman: Infinite 3D Human Creation with Precise Controlcited by 3
  25. 2025Jointly Understand Your Command and Intention:Reciprocal Co-Evolution between Scene-Aware 3D Human Motion Synthesis and Analysiscited by 2
  26. 2025Multi-modal Mutual-Guidance Conditional Prompt Learning for Vision-Language Modelscited by 2
  27. 2023ViT-Lens: Initiating Omni-Modal Exploration through 3D Insightscited by 1
  28. 2024Are Pose Estimators Ready for the Open World? STAGE: A GenAI Toolkit for Auditing 3D Human Pose Estimatorscited by 1
  29. 2026RAGMesh with FaME-G2E: Long-Form Text-Driven 3D Face Generation and Editingcited by 0