# Human-Art: A Versatile Human-Centric Dataset Bridging Natural and Artificial Scenes

arXiv: https://arxiv.org/abs/2303.02760 · HTML: https://mlsanity.com/paper/2303.02760/

## Key facts

- Cited by 68 papers in the map (data through 14 Oct 2026).
- Builds on 73 of its 99 references that are in the map.
- First posted on arXiv 5 Mar 2023.

## Builds on (most cited first, 30 of 73)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [GLIGEN: Open-Set Grounded Text-to-Image Generation](https://mlsanity.com/paper/2301.07093/) (2023)
- [First Order Motion Model for Image Animation](https://mlsanity.com/paper/2003.00196/) (2020)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation](https://mlsanity.com/paper/2204.12484/) (2022)
- [Learning from Synthetic Humans](https://mlsanity.com/paper/1701.01370/) (2017)
- [Everybody Dance Now](https://mlsanity.com/paper/1808.07371/) (2018)

## Built on by (most cited first, 30 of 68)

- [InstructDiffusion: A Generalist Modeling Interface for Vision Tasks](https://mlsanity.com/paper/2309.03895/) (2023)
- [VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Tasks](https://mlsanity.com/paper/2406.08394/) (2024)
- [HumanSD: A Native Skeleton-Guided Diffusion Model for Human Image Generation](https://mlsanity.com/paper/2304.04269/) (2023)
- [HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splatting](https://mlsanity.com/paper/2311.17061/) (2023)
- [DINO-X: A Unified Vision Model for Open-World Object Detection and Understanding](https://mlsanity.com/paper/2411.14347/) (2024)
- [OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generation](https://mlsanity.com/paper/2412.00115/) (2024)
- [Detect Anything via Next Point Prediction](https://mlsanity.com/paper/2510.12798/) (2025)
- [HyperHuman: Hyper-Realistic Human Generation with Latent Structural Diffusion](https://mlsanity.com/paper/2310.08579/) (2023)
- [CosmicMan: A Text-to-Image Foundation Model for Humans](https://mlsanity.com/paper/2404.01294/) (2024)
- [HumanRefiner: Benchmarking Abnormal Human Generation and Refining with Coarse-to-fine Pose-Reversible Guidance](https://mlsanity.com/paper/2407.06937/) (2024)
- [A Comprehensive Survey on Human Video Generation: Challenges, Methods, and Insights](https://mlsanity.com/paper/2407.08428/) (2024)
- [CoMusion: Towards Consistent Stochastic Human Motion Prediction via Motion Diffusion](https://mlsanity.com/paper/2305.12554/) (2023)
- [X-Pose: Detecting Any Keypoints](https://mlsanity.com/paper/2310.08530/) (2023)
- [Referring to Any Person](https://mlsanity.com/paper/2503.08507/) (2025)
- [RTMW: Real-Time Multi-Person 2D and 3D Whole-body Pose Estimation](https://mlsanity.com/paper/2407.08634/) (2024)
- [Towards Effective Usage of Human-Centric Priors in Diffusion Models for Text-based Human Image Generation](https://mlsanity.com/paper/2403.05239/) (2024)
- [LocLLM: Exploiting Generalizable Human Keypoint Localization via Large Language Model](https://mlsanity.com/paper/2406.04659/) (2024)
- [Diffusion-Based Visual Art Creation: A Survey and New Perspectives](https://mlsanity.com/paper/2408.12128/) (2024)
- [VirtualModel: Generating Object-ID-retentive Human-object Interaction Image by Diffusion Model for E-commerce Marketing](https://mlsanity.com/paper/2405.09985/) (2024)
- [GRPose: Learning Graph Relations for Human Image Generation with Pose Priors](https://mlsanity.com/paper/2408.16540/) (2024)
- [Neural Interactive Keypoint Detection](https://mlsanity.com/paper/2308.10174/) (2023)
- [Stable-Pose: Leveraging Transformers for Pose-Guided Text-to-Image Generation](https://mlsanity.com/paper/2406.02485/) (2024)
- [Nonisotropic Gaussian Diffusion for Realistic 3D Human Motion Prediction](https://mlsanity.com/paper/2501.06035/) (2025)
- [You Only Learn One Query: Learning Unified Human Query for Single-Stage Multi-Person Multi-Task Human-Centric Perception](https://mlsanity.com/paper/2312.05525/) (2023)
- [FastVideoEdit: Leveraging Consistency Models for Efficient Text-to-Video Editing](https://mlsanity.com/paper/2403.06269/) (2024)
- [RHanDS: Refining Malformed Hands for Generated Images with Decoupled Structure and Style Guidance](https://mlsanity.com/paper/2404.13984/) (2024)
- [MultiRef: Controllable Image Generation with Multiple Visual References](https://mlsanity.com/paper/2508.06905/) (2025)
- [FaceScore: Benchmarking and Enhancing Face Quality in Human Generation](https://mlsanity.com/paper/2406.17100/) (2024)
- [Vision as Unified Multimodal Generation](https://mlsanity.com/paper/2607.06560/) (2026)
- [FineControlNet: Fine-level Text Control for Image Generation with Spatially Aligned Text Control Injection](https://mlsanity.com/paper/2312.09252/) (2023)

## Abstract (arXiv)

Humans have long been recorded in a variety of forms since antiquity. For example, sculptures and paintings were the primary media for depicting human beings before the invention of cameras. However, most current human-centric computer vision tasks like human pose estimation and human image generation focus exclusively on natural images in the real world. Artificial humans, such as those in sculptures, paintings, and cartoons, are commonly neglected, making existing models fail in these scenarios. As an abstraction of life, art incorporates humans in both natural and artificial scenes. We take advantage of it and introduce the Human-Art dataset to bridge related tasks in natural and artificial scenarios. Specifically, Human-Art contains 50k high-quality images with over 123k person instances from 5 natural and 15 artificial scenarios, which are annotated with bounding boxes, keypoints, self-contact points, and text information for humans represented in both 2D and 3D. It is, therefore, comprehensive and versatile for various downstream tasks. We also provide a rich set of baseline results and detailed analyses for related tasks, including human detection, 2D and 3D human pose estimation, image generation, and motion transfer. As a challenging dataset, we hope Human-Art can provide insights for relevant research and open up new research questions.