mlsanityData to 7 Oct 2026

NToP: NeRF-Powered Large-scale Dataset Generation for 2D and 3D Human Pose Estimation in Top-View Fisheye Images

arXiv 2402.18196 ↗First posted 28 Feb 2024cs.CV cs.GRcited by 2

Not on the map. Why?

Abstract (arXiv)

Human pose estimation (HPE) in the top-view using fisheye cameras presents a promising and innovative application domain. However, the availability of datasets capturing this viewpoint is extremely limited, especially those with high-quality 2D and 3D keypoint annotations. Addressing this gap, we leverage the capabilities of Neural Radiance Fields (NeRF) technique to establish a comprehensive pipeline for generating human pose datasets from existing 2D and 3D datasets, specifically tailored for the top-view fisheye perspective. Through this pipeline, we create a novel dataset NToP570K (NeRF-powered Top-view human Pose dataset for fisheye cameras with over 570 thousand images), and conduct an extensive evaluation of its efficacy in enhancing neural networks for 2D and 3D top-view human pose estimation. A pretrained ViTPose-B model achieves an improvement in AP of 33.3 % on our validation set for 2D HPE after finetuning on our training set. A similarly finetuned HybrIK-Transformer model gains 53.7 mm reduction in PA-MPJPE for 3D HPE on the validation set.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 52 of its 78 references that are in the map. First posted on arXiv 28 Feb 2024.

Builds on

52 of 78 references.

References · 52

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2023Segment Anythingcited by 8,158
  4. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  5. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  6. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  7. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  8. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
  9. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 913
  10. 2020A Survey on Visual Transformercited by 653
  11. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  12. 2018Simple Baselines for Human Pose Estimation and Trackingcited by 502
38 more
  1. 2016Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervisioncited by 493
  2. 2022HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Videocited by 483
  3. 2022ViTPose: Simple Vision Transformer Baselines for Human Pose Estimationcited by 450
  4. 2017A simple yet effective baseline for 3d human pose estimationcited by 401
  5. 2020HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimationcited by 304
  6. 2019HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimationcited by 247
  7. 2022NeuMan: Neural Human Radiance Field from a Single Videocited by 244
  8. 2021A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Posecited by 233
  9. 2016Associative Embedding: End-to-End Learning for Joint Detection and Groupingcited by 211
  10. 2021Neural Articulated Radiance Fieldcited by 196
  11. 2022MotionBERT: A Unified Perspective on Learning Human Motion Representationscited by 188
  12. 2022TAVA: Template-free Animatable Volumetric Actorscited by 172
  13. 2019Learnable Triangulation of Human Posecited by 160
  14. 2021H-NeRF: Neural Radiance Fields for Rendering and Temporal Reconstruction of Humans in Motioncited by 157
  15. 2021HRFormer: High-Resolution Transformer for Dense Predictioncited by 149
  16. 2022InstantAvatar: Learning Avatars from Monocular Video in 60 Secondscited by 138
  17. 2023Vid2Avatar: 3D Avatar Reconstruction from Videos in the Wild via Self-supervised Scene Decompositioncited by 134
  18. 2020TransPose: Keypoint Localization via Transformercited by 126
  19. 2015DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimationcited by 117
  20. 2021TokenPose: Learning Keypoint Tokens for Human Pose Estimationcited by 106
  21. 2022Recovering 3D Human Mesh from Monocular Images: A Surveycited by 103
  22. 2023DNA-Rendering: A Diverse Neural Actor Repository for High-Fidelity Human-centric Renderingcited by 103
  23. 2021imGHUM: Implicit Generative Models of 3D Human Shape and Articulated Posecited by 102
  24. 2023SHERF: Generalizable Human NeRF from a Single Imagecited by 88
  25. 2023AvatarReX: Real-time Expressive Full-body Avatarscited by 83
  26. 2023Learning Neural Volumetric Representations of Dynamic Humans in Minutescited by 70
  27. 2022Fast-SNARF: A Fast Deformer for Articulated Neural Fieldscited by 55
  28. 2021Recent Advances in Monocular 2D and 3D Human Pose Estimation: A Deep Learning Perspectivecited by 49
  29. 2022Generalizable Neural Performer: Learning Robust Radiance Fields for Human Novel View Synthesiscited by 40
  30. 2023GM-NeRF: Learning Generalizable Model-based Neural Radiance Fields from Multi-view Imagescited by 24
  31. 2021DECA: Deep viewpoint-Equivariant human pose estimation using Capsule Autoencoderscited by 23
  32. 2023ActorsNeRF: Animatable Few-shot Human Rendering with Generalizable NeRFscited by 21
  33. 2022PLIKS: A Pseudo-Linear Inverse Kinematic Solver for 3D Human Body Estimationcited by 18
  34. 2023Unsupervised Volumetric Animationcited by 18
  35. 2020RAPiD: Rotation-Aware People Detection in Overhead Fisheye Imagescited by 17
  36. 2021OmniFlow: Human Omnidirectional Optical Flowcited by 9
  37. 2019Large Area 3D Human Pose Detection Via Stereo Reconstruction in Panoramic Camerascited by 4
  38. 2022OmniPD: One-Step Person Detection in Top-View Omnidirectional Indoor Scenescited by 3

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Systematic Comparison of Projection Methods for Monocular 3D Human Pose Estimation on Fisheye Imagescited by 1
  2. 2026PMOF: A Dataset and Benchmark for Passenger Monitoring Using Overhead Fisheye Camerascited by 0