mlsanityData to 7 Oct 2026

Tracking People by Predicting 3D Appearance, Location & Pose

arXiv 2112.04477 ↗First posted 8 Dec 2021cs.CVcited by 15

Not on the map. Why?

Abstract (arXiv)

In this paper, we present an approach for tracking people in monocular videos, by predicting their future 3D representations. To achieve this, we first lift people to 3D from a single frame in a robust way. This lifting includes information about the 3D pose of the person, his or her location in the 3D space, and the 3D appearance. As we track a person, we collect 3D observations over time in a tracklet representation. Given the 3D nature of our observations, we build temporal models for each one of the previous attributes. We use these models to predict the future state of the tracklet, including 3D location, 3D appearance, and 3D pose. For a future frame, we compute the similarity between the predicted state of a tracklet and the single frame observations in a probabilistic manner. Association is solved with simple Hungarian matching, and the matches are used to update the respective tracklets. We evaluate our approach on various benchmarks and report state-of-the-art results.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 35 of its 45 references that are in the map. First posted on arXiv 8 Dec 2021.

Builds on

35 of 45 references.

References · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017Mask R-CNNcited by 7,259
  3. 2017End-to-end Recovery of Human Shape and Posecited by 883
  4. 2016Performance Measures and a Data Set for Multi-Target, Multi-Camera Trackingcited by 685
  5. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 625
  6. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 615
  7. 2020HOTA: A Higher Order Metric for Evaluating Multi-Object Trackingcited by 546
  8. 2021TrackFormer: Multi-Object Tracking with Transformerscited by 515
  9. 2018Simple Baselines for Human Pose Estimation and Trackingcited by 502
  10. 2017AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actionscited by 449
  11. 2019Tracking without bells and whistlescited by 382
  12. 2016RMPE: Regional Multi-person Pose Estimationcited by 340
23 more
  1. 2021PARE: Part Attention Regressor for 3D Human Body Estimationcited by 334
  2. 2018Learning 3D Human Dynamics from Videocited by 315
  3. 2020MovieNet: A Holistic Dataset for Movie Understandingcited by 206
  4. 2017Single-Shot Multi-Person 3D Pose Estimation From Monocular RGBcited by 195
  5. 2017PoseTrack: A Benchmark for Human Pose Estimation and Trackingcited by 173
  6. 2021Probabilistic Modeling for Human Mesh Recoverycited by 152
  7. 2020MOTChallenge: A Benchmark for Single-Camera Multiple Target Trackingcited by 134
  8. 2021SPEC: Seeing People in the Wild with an Estimated Cameracited by 134
  9. 2020A Spatio-temporal Transformer for 3D Human Motion Predictioncited by 133
  10. 2021Learning to Track with Object Permanencecited by 117
  11. 2020GNN3DMOT: Graph Neural Network for 3D Multi-Object Tracking with Multi-Feature Learningcited by 113
  12. 2019Deep Learning in Video Multi-Object Tracking: A Surveycited by 107
  13. 20203D Human Motion Estimation via Motion Compression and Refinementcited by 98
  14. 2018Pose Flow: Efficient Online Pose Trackingcited by 80
  15. 2019Spatial-Temporal Relation Networks for Multi-Object Trackingcited by 74
  16. 2019How To Train Your Deep Multi-Object Trackercited by 71
  17. 2020Human Mesh Recovery from Multiple Shotscited by 52
  18. 2017Detect-and-Track: Efficient Pose Estimation in Videoscited by 51
  19. 2019Predicting 3D Human Dynamics from Videocited by 51
  20. 20204D Association Graph for Realtime Multi-person Motion Capture Using Multiple Video Camerascited by 49
  21. 2020Detecting Invisible Peoplecited by 31
  22. 201915 Keypoints Is All You Needcited by 23
  23. 2021Tracking People with 3D Representationscited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 6; 2025: 2; 2026: 3.

15 papers.

Sort:

By area:

  1. 2023BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motioncited by 229
  2. 2024OmniRe: Omni Urban Scene Reconstructioncited by 159
  3. 2023TRACE: 5D Temporal Regression of Avatars with Dynamic Cameras in 3D Environmentscited by 89
  4. 2022Benchmarking and Analyzing 3D Human Pose and Shape Estimation Beyond Algorithmscited by 28
  5. 2024OmniNOCS: A unified NOCS dataset and model for 3D lifting of 2D objectscited by 22
  6. 2024Guess The Unseen: Dynamic 3D Scene Reconstruction from Partial 2D Glimpsescited by 17
  7. 2025CoMotion: Concurrent Multi-person 3D Motioncited by 13
  8. 2023Cinematic Behavior Transfer via NeRF-based Differentiable Filmingcited by 12
  9. 2024Interacted Object Grounding in Spatio-Temporal Human-Object Interactionscited by 6
  10. 2025PhysHMR: Learning Humanoid Control Policies from Vision for Physically Plausible Human Motion Reconstructioncited by 5
  11. 2024Ask, Pose, Unite: Scaling Data Acquisition for Close Interactions with Vision Language Modelscited by 1
  12. 2024Motion Diffusion-Guided 3D Global HMR from a Dynamic Cameracited by 1
3 more
  1. 2026AHAP: Reconstructing Arbitrary Humans from Arbitrary Perspectives with Geometric Priorscited by 1
  2. 2026AttriBE: Quantifying Attribute Expressivity in Body Embeddings for Recognition and Identificationcited by 0
  3. 2026Multi-THuMBS: Multi-person Tracking of 3D Human Meshes Beyond Video Shotscited by 0