mlsanityData to 7 Oct 2026

MAS: Multi-view Ancestral Sampling for 3D motion generation using 2D diffusion

arXiv 2310.14729 ↗First posted 23 Oct 2023cs.CV cs.GRcited by 37

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We introduce Multi-view Ancestral Sampling (MAS), a method for 3D motion generation, using 2D diffusion models that were trained on motions obtained from in-the-wild videos. As such, MAS opens opportunities to exciting and diverse fields of motion previously under-explored as 3D data is scarce and hard to collect. MAS works by simultaneously denoising multiple 2D motion sequences representing different views of the same 3D motion. It ensures consistency across all views at each diffusion step by combining the individual generations into a unified 3D sequence, and projecting it back to the original views. We demonstrate MAS on 2D pose data acquired from videos depicting professional basketball maneuvers, rhythmic gymnastic performances featuring a ball apparatus, and horse races. In each of these domains, 3D motion capture is arduous, and yet, MAS generates diverse and realistic 3D sequences. Unlike the Score Distillation approach, which optimizes each sample by repeatedly applying small fixes, our method uses a sampling process that was constructed for the diffusion framework. As we demonstrate, MAS avoids common issues such as out-of-domain sampling and mode-collapse. https://guytevet.github.io/mas-page/

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 42 of its 61 references that are in the map. First posted on arXiv 23 Oct 2023.

Builds on

42 of 61 references.

Closest ancestors · 9

Same direction, newest first

  1. 2023HumanNorm: Learning Normal Diffusion Model for High-quality and Realistic 3D Human Generation2 Oct
  2. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  3. 2023Points-to-3D: Bridging the Gap between Sparse Points and Shape-Controllable Text-to-3D Generation26 Jul
  4. 2023ARTIC3D: Learning Robust Articulated 3D Shapes from Noisy Web Image Collections7 Jun
  5. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  6. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  7. 2023Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation14 Mar
  8. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation1 Dec
  9. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 33

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  2. 2022Human Motion Diffusion Modelcited by 1,062
  3. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
  4. 2022YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectorscited by 978
  5. 2021ByteTrack: Multi-Object Tracking by Associating Every Detection Boxcited by 812
  6. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 768
  7. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 692
  8. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 615
  9. 2022TEMOS: Generating diverse human motions from textual descriptionscited by 486
  10. 2023Instruct-NeRF2NeRF: Editing 3D Scenes with Instructionscited by 482
  11. 2022ViTPose: Simple Vision Transformer Baselines for Human Pose Estimationcited by 450
  12. 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 438
21 more
  1. 2022MotionCLIP: Exposing Human Motion Generation to CLIP Spacecited by 406
  2. 2023Human Motion Diffusion as a Generative Priorcited by 354
  3. 2016The KIT Motion-Language Datasetcited by 334
  4. 2019Language2Pose: Natural Language Grounded Pose Forecastingcited by 257
  5. 2022MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesiscited by 218
  6. 2022FLAME: Free-form Language-based Motion Synthesis & Editingcited by 217
  7. 2022AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Timecited by 199
  8. 2023Tracking Everything Everywhere All at Oncecited by 190
  9. 2022MotionBERT: A Unified Perspective on Learning Human Motion Representationscited by 188
  10. 2023Delta Denoising Scorecited by 120
  11. 2022MagicPony: Learning Articulated 3D Animals in the Wildcited by 111
  12. 2023Diffusion-Based 3D Human Pose Estimation with Multi-Hypothesis Aggregationcited by 81
  13. 2023Single Motion Diffusioncited by 65
  14. 2022MoDi: Unconditional Motion Synthesis from Diverse Datacited by 59
  15. 2023Make-An-Animation: Large-Scale Text-conditional 3D Human Motion Generationcited by 56
  16. 2022APT-36K: A Large-scale Benchmark for Animal Pose Estimation and Trackingcited by 55
  17. 2023GLA-GCN: Global-local Adaptive Graph Convolutional Network for 3D Human Pose Estimation from Monocular Videocited by 46
  18. 2018Can 3D Pose be Learned from 2D Projections Alone?cited by 44
  19. 2021ElePose: Unsupervised 3D Human Pose Estimation by Predicting Camera Elevation and Learning Normalizing Flows on 2D Posescited by 35
  20. 2022PLIKS: A Pseudo-Linear Inverse Kinematic Solver for 3D Human Body Estimationcited by 18
  21. 2021SVMAC: Unsupervised 3D Human Pose Estimation from a Single Image with Single-view-multi-angle Consistencycited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 13; 2025: 14; 2026: 8.

37 papers.

Sort:

By area:

  1. 2024Collaborative Video Diffusion: Consistent Multi-video Generation with Camera Controlcited by 117
  2. 2024CLoSD: Closing the Loop between Simulation and Diffusion for multi-task character controlcited by 103
  3. 2024DartControl: A Diffusion-Based Autoregressive Motion Model for Real-Time Text-Driven Motion Controlcited by 80
  4. 2024Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregressioncited by 78
  5. 2023PhysHOI: Physics-Based Imitation of Dynamic Human-Object Interactioncited by 72
  6. 2024Flexible Motion In-betweening with Diffusion Modelscited by 69
  7. 2025AnyTop: Character Animation Diffusion with Any Topologycited by 37
  8. 2024Monkey See, Monkey Do: Harnessing Self-attention in Motion Diffusion for Zero-shot Motion Transfercited by 30
  9. 2025Absolute Coordinates Make Motion Generation Easycited by 27
  10. 2024AvatarGO: Zero-shot 4D Human-Object Interaction Generation and Animationcited by 25
  11. 2023Ponymation: Learning Articulated 3D Animal Motions from Unlabeled Online Videoscited by 21
  12. 2024Consolidating Attention Features for Multi-view Image Editingcited by 19
25 more
  1. 2024Pay Attention and Move Better: Harnessing Attention for Interactive Motion Generation and Training-free Editingcited by 17
  2. 2025Dance Like a Chicken: Low-Rank Stylization for Human Motion Diffusioncited by 16
  3. 2025Sketch2Anim: Towards Transferring Sketch Storyboards into 3D Animationcited by 16
  4. 2025AnimaX: Animating the Inanimate in 3D with Joint Video-Pose Diffusion Modelscited by 15
  5. 2024Lifting Motion to the 3D World via 2D Diffusioncited by 11
  6. 2024Motion-2-To-3: Leveraging 2D Motion Data for 3D Motion Generationscited by 9
  7. 2025DIMO: Diverse 3D Motion Generation for Arbitrary Objectscited by 6
  8. 2025How to Move Your Dragon: Text-to-Motion Synthesis for Large-Vocabulary Objectscited by 5
  9. 2025MotionVerse: A Unified Multimodal Framework for Motion Comprehension, Generation and Editingcited by 5
  10. 2024CryoFM: A Flow-based Foundation Model for Cryo-EM Densitiescited by 4
  11. 2025Generative Motion Infilling From Imprecisely Timed Keyframescited by 4
  12. 2025Learning to Control Physically-simulated 3D Characters via Generating and Mimicking 2D Motionscited by 2
  13. 2026AnyLift: Scaling Motion Reconstruction from Internet Videos via 2D Diffusioncited by 2
  14. 2025PoseBH: Prototypical Multi-Dataset Training Beyond Human Pose Estimationcited by 1
  15. 2025Bimanual 3D Hand Motion and Articulation Forecasting in Everyday Imagescited by 1
  16. 2026Learning Context-Adaptive Motion Priors for Masked Motion Diffusion Models with Efficient Kinematic Attention Aggregationcited by 1
  17. 2026Self-Intersection-Aware 3D Human Motion Generation Using an Efficient Human Sphere Proxycited by 1
  18. 2024MotionDreamer: Exploring Semantic Video Diffusion features for Zero-Shot 3D Mesh Animationcited by 0
  19. 2025Mocap-2-to-3: Multi-view Lifting for Monocular Motion Recovery with 2D Pretrainingcited by 0
  20. 2025LaxMotion: Rethinking Supervision Granularity for 3D Human Motion Generationcited by 0
  21. 2026InterEdit: Navigating Text-Guided 3D Dyadic Human Motion Editingcited by 0
  22. 2026EgoMotion: Hierarchical Reasoning and Diffusion for Egocentric Vision-Language Motion Generationcited by 0
  23. 2026Unsupervised 3D Human Pose Estimation via Conditional Multi-view Ancestral Samplingcited by 0
  24. 2026VideoMDM: Towards 3D Human Motion Generation From 2D Supervisioncited by 0
  25. 2026Two2Four: Generative Quadruped Puppeteering from Human Motioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Modiff: Action-Conditioned 3D Motion Generation with Denoising Diffusion Probabilistic Models
  2. 2026Human Video Generation from a Single Image with 3D Pose and View Control
  3. 2026Variance Reduction on the Camera Axis: Multi-View Score Distillation for 3D
  4. 2026World2Motion: Turning Video World Models into 3D Human Motion Generators
  5. 2024ViMo: Generating Motions from Casual Videos
  6. 2026Ani3DHuman: Photorealistic 3D Human Animation with Self-guided Stochastic Sampling
  7. 2025Structure From Tracking: Distilling Structure-Preserving Motion for Video Generation
  8. 2024Diffusion2: Dynamic 3D Content Generation via Score Composition of Video and Multi-view Diffusion Models