mlsanityData to 14 Oct 2026

ReMoS: 3D Motion-Conditioned Reaction Synthesis for Two-Person Interactions

arXiv 2311.17057 ↗First posted 28 Nov 2023cs.CVcited by 62

Not on the map. Why?

Abstract (arXiv)

Current approaches for 3D human motion synthesis generate high quality animations of digital humans performing a wide variety of actions and gestures. However, a notable technological gap exists in addressing the complex dynamics of multi human interactions within this paradigm. In this work, we present ReMoS, a denoising diffusion based model that synthesizes full body reactive motion of a person in a two person interaction scenario. Given the motion of one person, we employ a combined spatio temporal cross attention mechanism to synthesize the reactive body and hand motion of the second person, thereby completing the interactions between the two. We demonstrate ReMoS across challenging two person scenarios such as pair dancing, Ninjutsu, kickboxing, and acrobatics, where one persons movements have complex and diverse influences on the other. We also contribute the ReMoCap dataset for two person interactions containing full body and finger motions. We evaluate ReMoS through multiple quantitative metrics, qualitative visualizations, and a user study, and also indicate usability in interactive motion editing applications.

Key facts. Cited by 62 papers in the map (data through 14 Oct 2026). Builds on 51 of its 87 references that are in the map. First posted on arXiv 28 Nov 2023.

Builds on

51 of 87 references.

References · 51

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2020Denoising Diffusion Implicit Modelscited by 9,781
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  7. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,163
  8. 2022Human Motion Diffusion Modelcited by 1,080
  9. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 950
  10. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 780
  11. 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 588
  12. 2019NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understandingcited by 533
38 more
  1. 2022TEMOS: Generating diverse human motions from textual descriptionscited by 492
  2. 2022PhysDiff: Physics-Guided Human Motion Diffusion Modelcited by 368
  3. 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 359
  4. 2023Human Motion Diffusion as a Generative Priorcited by 357
  5. 2022EDGE: Editable Dance Generation From Musiccited by 287
  6. 2023Diffusion-based Generation, Optimization, and Planning in 3D Scenescited by 239
  7. 2023OmniControl: Control Any Joint at Any Time for Human Motion Generationcited by 238
  8. 2023Guided Motion Diffusion for Controllable Human Motion Synthesiscited by 223
  9. 2022MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesiscited by 218
  10. 2023Object Motion Guided Human Motion Synthesiscited by 212
  11. 2023InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactionscited by 206
  12. 2023CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Priorcited by 184
  13. 2023InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusioncited by 182
  14. 2021Synthesis of Compositional Animations from Textual Descriptionscited by 178
  15. 2022TEACH: Temporal Action Composition for 3D Humanscited by 176
  16. 2018Understanding Batch Normalizationcited by 159
  17. 2023GestureDiffuCLIP: Gesture Diffusion Model with CLIP Latentscited by 158
  18. 2023Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusioncited by 138
  19. 2022COUCH: Towards Controllable Human-Chair Interactionscited by 135
  20. 2023Taming Diffusion Models for Audio-Driven Co-Speech Gesture Generationcited by 129
  21. 2023State of the Art on Diffusion Models for Visual Computingcited by 127
  22. 2022Learning to Listen: Modeling Non-Deterministic Dyadic Facial Motioncited by 126
  23. 2022IMos: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactionscited by 124
  24. 2018Robots Learn Social Skills: End-to-End Learning of Co-Speech Gesture Generation for Humanoid Robotscited by 110
  25. 2023Affordance Diffusion: Synthesizing Hand-Object Interactionscited by 110
  26. 2021Text2Gestures: A Transformer-Based Network for Generating Emotive Body Gestures for Virtual Agentscited by 106
  27. 2023NIFTY: Neural Object Interaction Fields for Guided Human Motion Synthesiscited by 78
  28. 2021Multi-Person Extreme Motion Predictioncited by 75
  29. 2021Neural Monocular 3D Human Motion Capture with Physical Awarenesscited by 71
  30. 2022UDE: A Unified Driving Engine for Human Motion Generationcited by 67
  31. 2021Multi-Person 3D Motion Prediction with Multi-Range Transformerscited by 63
  32. 2021Speech2AffectiveGestures: Synthesizing Co-Speech Gestures with Generative Adversarial Affective Expression Learningcited by 58
  33. 2021Rhythm is a Dancer: Music-Driven Motion Synthesis with Global Structurecited by 53
  34. 2022Interaction Transformer for Human Reaction Generationcited by 42
  35. 2019To React or not to React: End-to-End Visual Pose Forecasting for Personalized Avatar during Dyadic Conversationscited by 26
  36. 2023ROAM: Robust and Object-Aware Motion Generation Using Neural Pose Descriptorscited by 23
  37. 2021GAN-based Reactive Motion Synthesis with Class-aware Discriminators for Human-human Interactioncited by 18
  38. 2023Bipartite Graph Diffusion Model for Human Interaction Generationcited by 11

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 10; 2025: 30; 2026: 22.

62 papers.

Sort:

By area:

  1. 2025InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactionscited by 72
  2. 2024InterMask: 3D Human Interaction Generation via Collaborative Masked Modelingcited by 47
  3. 2024BimArt: A Unified Approach for the Synthesis of 3D Bimanual Interaction with Articulated Objectscited by 30
  4. 2025Ready-to-React: Online Reaction Policy for Two-Character Interaction Generationcited by 28
  5. 2025SimMotionEdit: Text-Based Human Motion Editing with Motion Similarity Predictioncited by 27
  6. 2024Retrieving Semantics from the Deep: an RAG Solution for Gesture Synthesiscited by 26
  7. 2025A Survey on Human Interaction Motion Generationcited by 25
  8. 2025DuetGen: Music Driven Two-Person Dance Generation via Hierarchical Masked Modelingcited by 21
  9. 2025HERO: Human Reaction Generation from Videoscited by 15
  10. 2025SocialGen: Modeling Multi-Human Social Interaction with Language Modelscited by 14
  11. 2024SCENIC: Scene-aware Semantic Navigation with Instruction-guided Controlcited by 13
  12. 2025Shape My Moves: Text-Driven Shape-Aware Synthesis of Human Motionscited by 11
38 more
  1. 2024Synergy and Synchrony in Couple Dancescited by 9
  2. 2024You Think, You ACT: The New Task of Arbitrary Text to Motion Generationcited by 8
  3. 2025Multi-Person Interaction Generation from Two-Person Motion Priorscited by 8
  4. 2024FORCE: Physics-aware Human-object Interactioncited by 7
  5. 20253D Human Interaction Generation: A Surveycited by 7
  6. 2026DyaDiT: A Multi-Modal Diffusion Transformer for Socially Favorable Dyadic Gesture Generationcited by 7
  7. 2025Unified Multi-Modal Interactive & Reactive 3D Motion Generation via Rectified Flowcited by 6
  8. 2025Interact2Ar: Full-Body Human-Human Interaction Generation via Autoregressive Diffusion Modelscited by 6
  9. 2024Two-Person Interaction Augmentation with Skeleton Priorscited by 5
  10. 2024A Unified Framework for Motion Reasoning and Generation in Human Interactioncited by 5
  11. 2025MotionVerse: A Unified Multimodal Framework for Motion Comprehension, Generation and Editingcited by 5
  12. 2026TeamHOI: Learning a Unified Policy for Cooperative Human-Object Interactions with Any Team Sizecited by 5
  13. 2026Omni-Supervised Motion Editing: Balancing Change and Invariance through Positive-Negative Learningcited by 5
  14. 2025PhysiInter: Integrating Physical Mapping for High-Fidelity Human Interaction Generationcited by 4
  15. 2025Event-Driven Storytelling with Multiple Lifelike Humans in a 3D Scenecited by 4
  16. 2025MDD: A Dataset for Text-and-Music Conditioned Duet Dance Generationcited by 4
  17. 2025Diffusion Forcing for Multi-Agent Interaction Sequence Modelingcited by 4
  18. 2026HINT: Hierarchical Interaction Modeling for Autoregressive Multi-Human Motion Generationcited by 4
  19. 2024SMGDiff: Soccer Motion Generation using diffusion probabilistic modelscited by 3
  20. 2025Leader and Follower: Interactive Motion Generation under Trajectory Constraintscited by 3
  21. 2026SceMoS: Scene-Aware 3D Human Motion Synthesis by Planning with Geometry-Grounded Tokenscited by 3
  22. 2026Disentangled Hierarchical VAE for 3D Human-Human Interaction Generationcited by 3
  23. 2026PolySLGen: Online Multimodal Speaking-Listening Reaction Generation in Polyadic Interactioncited by 3
  24. 2025Betsu-Betsu: Multi-View Separable 3D Reconstruction of Two Interacting Objectscited by 2
  25. 2025CoMPAS3D: A Dataset and Benchmark for Interactive Motioncited by 2
  26. 2025E-React: Towards Emotionally Controlled Synthesis of Human Reactionscited by 2
  27. 2025InterAct: A Large-Scale Dataset of Dynamic, Expressive and Interactive Activities between Two People in Daily Scenarioscited by 2
  28. 2025MoReact: Generating Reactive Motion from Textual Descriptionscited by 2
  29. 2025Learning to Generate Human-Human-Object Interactions from Textual Descriptionscited by 2
  30. 2025MoLingo: Motion-Language Alignment for Text-to-Human Motion Generationcited by 2
  31. 2026DyaPlex: Full-Duplex Speech-Motion Model for Dyadic Interactioncited by 2
  32. 2025Poly-Autoregressive Prediction for Modeling Interactionscited by 1
  33. 2025ReactDance: Hierarchical Representation for High-Fidelity and Coherent Long-Form Reactive Dance Generationcited by 1
  34. 2025ScanMove: Motion Prediction and Transfer for Unregistered Body Meshescited by 1
  35. 2026ReactMotion: Generating Reactive Listener Motions from Speaker Utterancecited by 1
  36. 2026ReMoGen: Real-time Human Interaction-to-Reaction Generation via Modular Learning from Diverse Datacited by 1
  37. 2026MOCHI: Motion Enhancement of Collaborative Human-object Interactionscited by 1
  38. 2026Social Structure Matters in 3D Human-Human Interaction Generationcited by 1

50 most cited of 62 All 62 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Contact Matrix: Enhancing Dance Motion Synthesis with Precise Interaction Modeling
  2. 2024Two-in-One: Unified Multi-Person Interactive Motion Generation by Latent Diffusion Transformer
  3. 2026Dex2HOI: Dexterous Bimanual Two-Object Interaction Generation
  4. 2025Re-HOLD: Video Hand Object Interaction Reenactment via adaptive Layout-instructed Diffusion Model
  5. 2023InterControl: Zero-shot Human Interaction Generation by Controlling Every Joint
  6. 2024SyncDiff: Synchronized Motion Diffusion for Multi-Body Human-Object Interaction Synthesis
  7. 2024It Takes Two: Real-time Co-Speech Two-person's Interaction Generation via Reactive Auto-regressive Diffusion Model
  8. 2022Pretrained Diffusion Models for Unified Human Motion Synthesis