# PaCMO: Partner Dependent Human Motion Generation in Dyadic Human Activity using Neural Operators

arXiv: https://arxiv.org/abs/2211.16210 · HTML: https://mlsanity.com/paper/2211.16210/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 54 references that are in the map.
- First posted on arXiv 25 Nov 2022.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding](https://mlsanity.com/paper/1905.04757/) (2019)
- [Action-Conditioned 3D Human Motion Synthesis with Transformer VAE](https://mlsanity.com/paper/2104.05670/) (2021)
- [TEMOS: Generating diverse human motions from textual descriptions](https://mlsanity.com/paper/2204.14109/) (2022)
- [Action2Motion: Conditioned Generation of 3D Human Motions](https://mlsanity.com/paper/2007.15240/) (2020)
- [Neural Operator: Learning Maps Between Function Spaces](https://mlsanity.com/paper/2108.08481/) (2021)
- [MotionCLIP: Exposing Human Motion Generation to CLIP Space](https://mlsanity.com/paper/2203.08063/) (2022)
- [U-FNO -- An enhanced Fourier neural operator-based deep-learning model for multiphase flow](https://mlsanity.com/paper/2109.03697/) (2021)
- [Language2Pose: Natural Language Grounded Pose Forecasting](https://mlsanity.com/paper/1907.01108/) (2019)
- [U-NO: U-shaped Neural Operators](https://mlsanity.com/paper/2204.11127/) (2022)
- [TEACH: Temporal Action Composition for 3D Humans](https://mlsanity.com/paper/2209.04066/) (2022)
- [GOAL: Generating 4D Whole-Body Motion for Hand-Object Grasping](https://mlsanity.com/paper/2112.11454/) (2021)
- [Text2Action: Generative Adversarial Synthesis from Language to Action](https://mlsanity.com/paper/1710.05298/) (2017)
- [LCR-Net++: Multi-person 2D and 3D Pose Detection in Natural Images](https://mlsanity.com/paper/1803.00455/) (2018)
- [MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamics](https://mlsanity.com/paper/1808.04545/) (2018)
- [NeMF: Neural Motion Fields for Kinematic Animation](https://mlsanity.com/paper/2206.03287/) (2022)
- [Generative Adversarial Neural Operators](https://mlsanity.com/paper/2205.03017/) (2022)

## Built on by (most cited first, 8 of 8)

- [Universal Functional Regression with Neural Operator Flows](https://mlsanity.com/paper/2404.02986/) (2024)
- [Neural Operator: Is data all you need to model the world? An insight into the paradigm of data-driven scientific ML](https://mlsanity.com/paper/2301.13331/) (2023)
- [Unified Multi-Modal Interactive & Reactive 3D Motion Generation via Rectified Flow](https://mlsanity.com/paper/2509.24099/) (2025)
- [M3Act: Learning from Synthetic Human Group Activities](https://mlsanity.com/paper/2306.16772/) (2023)
- [On the Equivalency, Substitutability, and Flexibility of Synthetic Data](https://mlsanity.com/paper/2403.16244/) (2024)
- [Learning Lagrangian Interaction Dynamics with Sampling-Based Model Order Reduction](https://mlsanity.com/paper/2407.03925/) (2024)
- [Personalized Pose Forecasting](https://mlsanity.com/paper/2312.03528/) (2023)
- [HessianForge: Scalable LiDAR reconstruction with Physics-Informed Neural Representation and Smoothness Energy Constraints](https://mlsanity.com/paper/2503.08929/) (2025)

## Abstract (arXiv)

We address the problem of generating 3D human motions in dyadic activities. In contrast to the concurrent works, which mainly focus on generating the motion of a single actor from the textual description, we generate the motion of one of the actors from the motion of the other participating actor in the action. This is a particularly challenging, under-explored problem, that requires learning intricate relationships between the motion of two actors participating in an action and also identifying the action from the motion of one actor. To address these, we propose partner conditioned motion operator (PaCMO), a neural operator-based generative model which learns the distribution of human motion conditioned by the partner's motion in function spaces through adversarial training. Our model can handle long unlabeled action sequences at arbitrary time resolution. We also introduce the "Functional Frechet Inception Distance" (F2ID) metric for capturing similarity between real and generated data for function spaces. We test PaCMO on NTU RGB+D and DuetDance datasets and our model produces realistic results evidenced by the F2ID score and the conducted user study.