# Pose for Everything: Towards Category-Agnostic Pose Estimation

arXiv: https://arxiv.org/abs/2207.10387 · HTML: https://mlsanity.com/paper/2207.10387/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 55 of its 76 references that are in the map.
- First posted on arXiv 21 Jul 2022.

## Builds on (most cited first, 30 of 55)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [Image Transformer](https://mlsanity.com/paper/1802.05751/) (2018)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [Cascaded Pyramid Network for Multi-Person Pose Estimation](https://mlsanity.com/paper/1711.07319/) (2017)
- [Convolutional Pose Machines](https://mlsanity.com/paper/1602.00134/) (2016)
- [DeepPose: Human Pose Estimation via Deep Neural Networks](https://mlsanity.com/paper/1312.4659/) (2013)
- [FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Images](https://mlsanity.com/paper/1909.04349/) (2019)
- [HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation](https://mlsanity.com/paper/1908.10357/) (2019)
- [InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image](https://mlsanity.com/paper/2008.09309/) (2020)
- [Learning to Estimate 3D Hand Pose from Single RGB Images](https://mlsanity.com/paper/1705.01389/) (2017)
- [Hand Keypoint Detection in Single Images using Multiview Bootstrapping](https://mlsanity.com/paper/1704.07809/) (2017)
- [CANet: Class-Agnostic Segmentation Networks with Iterative Refinement and Attentive Few-Shot Learning](https://mlsanity.com/paper/1903.02351/) (2019)
- [CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmark](https://mlsanity.com/paper/1812.00324/) (2018)
- [Low-Shot Learning from Imaginary Data](https://mlsanity.com/paper/1801.05401/) (2018)
- [Whole-Body Human Pose Estimation in the Wild](https://mlsanity.com/paper/2007.11858/) (2020)
- [PoseTrack: A Benchmark for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1710.10000/) (2017)
- [HRFormer: High-Resolution Transformer for Dense Prediction](https://mlsanity.com/paper/2110.09408/) (2021)
- [GANerated Hands for Real-time 3D Hand Tracking from Monocular RGB](https://mlsanity.com/paper/1712.01057/) (2017)
- [Look at Boundary: A Boundary-Aware Face Alignment Algorithm](https://mlsanity.com/paper/1805.10483/) (2018)

## Built on by (most cited first, 30 of 36)

- [AutoDecoding Latent 3D Diffusion Models](https://mlsanity.com/paper/2307.05445/) (2023)
- [X-Pose: Detecting Any Keypoints](https://mlsanity.com/paper/2310.08530/) (2023)
- [A Graph-Based Approach for Category-Agnostic Pose Estimation](https://mlsanity.com/paper/2311.17891/) (2023)
- [Segment Anything in Non-Euclidean Domains: Challenges and Opportunities](https://mlsanity.com/paper/2304.11595/) (2023)
- [FaceShot: Bring Any Character into Life](https://mlsanity.com/paper/2503.00740/) (2025)
- [Neural Interactive Keypoint Detection](https://mlsanity.com/paper/2308.10174/) (2023)
- [Open-Vocabulary Animal Keypoint Detection with Semantic-feature Matching](https://mlsanity.com/paper/2310.05056/) (2023)
- [MoCapAnything: Unified 3D Motion Capture for Arbitrary Skeletons from Monocular Videos](https://mlsanity.com/paper/2512.10881/) (2025)
- [You Only Learn One Query: Learning Unified Human Query for Single-Stage Multi-Person Multi-Task Human-Centric Perception](https://mlsanity.com/paper/2312.05525/) (2023)
- [CapeX: Category-Agnostic Pose Estimation from Textual Point Explanation](https://mlsanity.com/paper/2406.00384/) (2024)
- [OpenKD: Opening Prompt Diversity for Zero- and Few-shot Keypoint Detection](https://mlsanity.com/paper/2409.19899/) (2024)
- [UniAP: Towards Universal Animal Perception in Vision via Few-shot Learning](https://mlsanity.com/paper/2308.09953/) (2023)
- [Meta-Point Learning and Refining for Category-Agnostic Pose Estimation](https://mlsanity.com/paper/2403.13647/) (2024)
- [KptLLM: Unveiling the Power of Large Language Model for Keypoint Comprehension](https://mlsanity.com/paper/2411.01846/) (2024)
- [Meta Omnium: A Benchmark for General-Purpose Learning-to-Learn](https://mlsanity.com/paper/2305.07625/) (2023)
- [AnyMoLe: Any Character Motion In-betweening Leveraging Video Diffusion Models](https://mlsanity.com/paper/2503.08417/) (2025)
- [VoxelKeypointFusion: Generalizable Multi-View Multi-Person Pose Estimation](https://mlsanity.com/paper/2410.18723/) (2024)
- [Learning Structure-Supporting Dependencies via Keypoint Interactive Transformer for General Mammal Pose Estimation](https://mlsanity.com/paper/2502.18214/) (2025)
- [Probabilistic Prompt Distribution Learning for Animal Pose Estimation](https://mlsanity.com/paper/2503.16120/) (2025)
- [MARCO: Navigating the Unseen Space of Semantic Correspondence](https://mlsanity.com/paper/2604.18267/) (2026)
- [UniFS: Universal Few-shot Instance Perception with Point Representations](https://mlsanity.com/paper/2404.19401/) (2024)
- [CapeLLM: Support-Free Category-Agnostic Pose Estimation with Multimodal Large Language Models](https://mlsanity.com/paper/2411.06869/) (2024)
- [Recurrent Feature Mining and Keypoint Mixup Padding for Category-Agnostic Pose Estimation](https://mlsanity.com/paper/2503.21140/) (2025)
- [MoCapAnything V2: End-to-End Motion Capture for Arbitrary Skeletons](https://mlsanity.com/paper/2604.28130/) (2026)
- [FAFA: Frequency-Aware Flow-Aided Self-Supervision for Underwater Object Pose Estimation](https://mlsanity.com/paper/2409.16600/) (2024)
- [PoseBH: Prototypical Multi-Dataset Training Beyond Human Pose Estimation](https://mlsanity.com/paper/2505.17475/) (2025)
- [Edge Weight Prediction For Category-Agnostic Pose Estimation](https://mlsanity.com/paper/2411.16665/) (2024)
- [Skel3D: Skeleton Guided Novel View Synthesis](https://mlsanity.com/paper/2412.03407/) (2024)
- [Incremental Object Keypoint Learning](https://mlsanity.com/paper/2503.20248/) (2025)
- [KptLLM++: Towards Generic Keypoint Comprehension with Large Language Model](https://mlsanity.com/paper/2507.11102/) (2025)

## Abstract (arXiv)

Existing works on 2D pose estimation mainly focus on a certain category, e.g. human, animal, and vehicle. However, there are lots of application scenarios that require detecting the poses/keypoints of the unseen class of objects. In this paper, we introduce the task of Category-Agnostic Pose Estimation (CAPE), which aims to create a pose estimation model capable of detecting the pose of any class of object given only a few samples with keypoint definition. To achieve this goal, we formulate the pose estimation problem as a keypoint matching problem and design a novel CAPE framework, termed POse Matching Network (POMNet). A transformer-based Keypoint Interaction Module (KIM) is proposed to capture both the interactions among different keypoints and the relationship between the support and query images. We also introduce Multi-category Pose (MP-100) dataset, which is a 2D pose dataset of 100 object categories containing over 20K instances and is well-designed for developing CAPE algorithms. Experiments show that our method outperforms other baseline approaches by a large margin. Codes and data are available at https://github.com/luminxu/Pose-for-Everything.