mlsanityData to 7 Oct 2026

Snipper: A Spatiotemporal Transformer for Simultaneous Multi-Person 3D Pose Estimation Tracking and Forecasting on a Video Snippet

arXiv 2207.04320 ↗First posted 9 Jul 2022cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

Multi-person pose understanding from RGB videos involves three complex tasks: pose estimation, tracking and motion forecasting. Intuitively, accurate multi-person pose estimation facilitates robust tracking, and robust tracking builds crucial history for correct motion forecasting. Most existing works either focus on a single task or employ multi-stage approaches to solving multiple tasks separately, which tends to make sub-optimal decision at each stage and also fail to exploit correlations among the three tasks. In this paper, we propose Snipper, a unified framework to perform multi-person 3D pose estimation, tracking, and motion forecasting simultaneously in a single stage. We propose an efficient yet powerful deformable attention mechanism to aggregate spatiotemporal information from the video snippet. Building upon this deformable attention, a video transformer is learned to encode the spatiotemporal features from the multi-frame snippet and to decode informative pose features for multi-person pose queries. Finally, these pose queries are regressed to predict multi-person pose trajectories and future motions in a single shot. In the experiments, we show the effectiveness of Snipper on three challenging public datasets where our generic model rivals specialized state-of-art baselines for pose estimation, tracking, and forecasting.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 53 of its 79 references that are in the map. First posted on arXiv 9 Jul 2022.

Builds on

53 of 79 references.

References · 53

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2020End-to-End Object Detection with Transformerscited by 7,384
  4. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  5. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  6. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,161
  7. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 913
  8. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 687
  9. 2021TrackFormer: Multi-Object Tracking with Transformerscited by 515
  10. 2018Simple Baselines for Human Pose Estimation and Trackingcited by 502
  11. 2020End-to-End Video Instance Segmentation with Transformerscited by 443
  12. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 417
38 more
  1. 2021MOTR: End-to-End Multiple-Object Tracking with Transformercited by 379
  2. 2017Cascaded Pyramid Network for Multi-Person Pose Estimationcited by 341
  3. 20213D Human Pose Estimation with Spatial and Temporal Transformerscited by 271
  4. 2020Monocular, One-stage, Regression of Multiple 3D Peoplecited by 216
  5. 2016Associative Embedding: End-to-End Learning for Joint Detection and Groupingcited by 211
  6. 2017Single-Shot Multi-Person 3D Pose Estimation From Monocular RGBcited by 195
  7. 2020Long-term Human Motion Prediction with Scene Contextcited by 191
  8. 2016Panoptic Studio: A Massively Multiview System for Social Interaction Capturecited by 173
  9. 2017PoseTrack: A Benchmark for Human Pose Estimation and Trackingcited by 173
  10. 2021MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimationcited by 151
  11. 2018BodyNet: Volumetric Inference of 3D Human Body Shapescited by 141
  12. 2019Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Imagecited by 131
  13. 2015DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimationcited by 117
  14. 2019Single-Stage Multi-Person Pose Machinescited by 95
  15. 2021Graph Stacked Hourglass Networks for 3D Human Pose Estimationcited by 94
  16. 2017Learning to Generate Long-term Future via Hierarchical Predictioncited by 88
  17. 2017The Pose Knows: Video Forecasting by Generating Pose Futurescited by 88
  18. 2018LCR-Net++: Multi-person 2D and 3D Pose Detection in Natural Imagescited by 88
  19. 2018Learning to Detect and Track Visible and Occluded Body Joints in a Virtual Worldcited by 81
  20. 2019XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Cameracited by 68
  21. 2021Encoder-decoder with Multi-level Attention for 3D Human Shape and Pose Estimationcited by 62
  22. 2020SMAP: Single-Shot Multi-Person Absolute 3D Pose Estimationcited by 52
  23. 2017Detect-and-Track: Efficient Pose Estimation in Videoscited by 51
  24. 2020Combining detection and tracking for human pose estimation in videoscited by 49
  25. 2019In the Wild Human Pose Estimation Using Explicit 2D Features and Intermediate 3D Representationscited by 48
  26. 2019Towards Social Artificial Intelligence: Nonverbal Social Signal Prediction in A Triadic Interactioncited by 48
  27. 2020HMOR: Hierarchical Multi-Person Ordinal Relations for Monocular Multi-Person 3D Pose Estimationcited by 44
  28. 2021VoxelTrack: Multi-Person 3D Human Pose Estimation and Tracking in the Wildcited by 39
  29. 2020Compressed Volumetric Heatmaps for Multi-Person 3D Pose Estimationcited by 36
  30. 2020Richly Activated Graph Convolutional Network for Robust Skeleton-based Action Recognitioncited by 36
  31. 2018Efficient Online Multi-Person 2D Pose Tracking with Recurrent Spatio-Temporal Affinity Fieldscited by 32
  32. 2020Graph and Temporal Convolutional Networks for 3D Multi-person Pose Estimation in Monocular Videoscited by 23
  33. 2021PandaNet : Anchor-Based Single-Shot Multi-Person 3D Pose Estimationcited by 22
  34. 2021Tracking People with 3D Representationscited by 22
  35. 2021Monocular 3D Multi-Person Pose Estimation by Integrating Top-Down and Bottom-Up Networkscited by 16
  36. 2019Pose estimator and tracker using temporal flow maps for limbscited by 13
  37. 2021Human Pose and Shape Estimation from Single Polarization Imagescited by 13
  38. 2019View Invariant 3D Human Pose Estimationcited by 11

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 2; 2025: 0; 2026: 1.

6 papers.

  1. 2023EgoHumans: An Egocentric 3D Multi-Human Benchmarkcited by 58
  2. 2023TEMPO: Efficient Multi-View Pose Estimation, Tracking, and Forecastingcited by 22
  3. 2023Human Pose-based Estimation, Tracking and Action Recognition with Deep Learning: A Surveycited by 6
  4. 2024Simultaneously Recovering Multi-Person Meshes and Multi-View Cameras with Human Semanticscited by 3
  5. 2024Relation Learning and Aggregate-attention for Multi-person Motion Predictioncited by 0
  6. 2026Sound-based Multi-Person 3D Pose Estimationcited by 0