mlsanityData to 14 Oct 2026

Choreographing the Digital Canvas: A Machine Learning Approach to Artistic Performance

arXiv 2404.00054 ↗First posted 26 Mar 2024cs.HC cs.GR cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

This paper introduces the concept of a design tool for artistic performances based on attribute descriptions. To do so, we used a specific performance of falling actions. The platform integrates a novel machine-learning (ML) model with an interactive interface to generate and visualize artistic movements. Our approach's core is a cyclic Attribute-Conditioned Variational Autoencoder (AC-VAE) model developed to address the challenge of capturing and generating realistic 3D human body motions from motion capture (MoCap) data. We created a unique dataset focused on the dynamics of falling movements, characterized by a new ontology that divides motion into three distinct phases: Impact, Glitch, and Fall. The ML model's innovation lies in its ability to learn these phases separately. It is achieved by applying comprehensive data augmentation techniques and an initial pose loss function to generate natural and plausible motion. Our web-based interface provides an intuitive platform for artists to engage with this technology, offering fine-grained control over motion attributes and interactive visualization tools, including a 360-degree view and a dynamic timeline for playback manipulation. Our research paves the way for a future where technology amplifies the creative potential of human expression, making sophisticated motion generation accessible to a wider artistic community.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 20 of its 42 references that are in the map. First posted on arXiv 26 Mar 2024.

Builds on

20 of 42 references.

References · 20

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2022Generative Adversarial Networkscited by 5,627
  5. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  6. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,214
  7. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,209
  8. 2022Human Motion Diffusion Modelcited by 1,080
  9. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 780
  10. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 501
  11. 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 441
  12. 2022PhysDiff: Physics-Guided Human Motion Diffusion Modelcited by 368
8 more
  1. 2020STAR: Sparse Trained Articulated Human Body Regressorcited by 194
  2. 2017HP-GAN: Probabilistic 3D human motion prediction via GANcited by 176
  3. 2018A Neural Temporal Model for Human Motion Predictioncited by 87
  4. 2018MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamicscited by 86
  5. 2018Pose Guided Human Video Generationcited by 67
  6. 2018Human Motion Modeling using DVGANscited by 45
  7. 2018Combining Recurrent Neural Networks and Adversarial Training for Human Motion Synthesis and Controlcited by 39
  8. 2019A Large-scale Varying-view RGB-D Action Dataset for Arbitrary-view Human Action Recognitioncited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Text Dictates, Music Decorates: Energy-based Attention for Editable Dance Motion Generationcited by 0