# CLAMP: Prompt-based Contrastive Learning for Connecting Language and Animal Pose

arXiv: https://arxiv.org/abs/2206.11752 · HTML: https://mlsanity.com/paper/2206.11752/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 43 references that are in the map.
- First posted on arXiv 23 Jun 2022.

## Builds on (most cited first, 30 of 34)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Conditional Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2203.05557/) (2022)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Grounded Language-Image Pre-training](https://mlsanity.com/paper/2112.03857/) (2021)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments](https://mlsanity.com/paper/1711.07280/) (2017)
- [CLIP-Adapter: Better Vision-Language Models with Feature Adapters](https://mlsanity.com/paper/2110.04544/) (2021)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [DenseCLIP: Language-Guided Dense Prediction with Context-Aware Prompting](https://mlsanity.com/paper/2112.01518/) (2021)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation](https://mlsanity.com/paper/2204.12484/) (2022)
- [Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling](https://mlsanity.com/paper/2111.03930/) (2021)
- [Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigm](https://mlsanity.com/paper/2110.05208/) (2021)
- [RMPE: Regional Multi-person Pose Estimation](https://mlsanity.com/paper/1612.00137/) (2016)
- [Associative Embedding: End-to-End Learning for Joint Detection and Grouping](https://mlsanity.com/paper/1611.05424/) (2016)
- [MirrorGAN: Learning Text-to-image Generation by Redescription](https://mlsanity.com/paper/1903.05854/) (2019)
- [AP-10K: A Benchmark for Animal Pose Estimation in the Wild](https://mlsanity.com/paper/2108.12617/) (2021)
- [Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression](https://mlsanity.com/paper/2104.02300/) (2021)
- [Cross-Domain Adaptation for Animal Pose Estimation](https://mlsanity.com/paper/1908.05806/) (2019)
- [CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP](https://mlsanity.com/paper/2110.11316/) (2021)
- [Learning from Synthetic Animals](https://mlsanity.com/paper/1912.08265/) (2019)
- [From Synthetic to Real: Unsupervised Domain Adaptation for Animal Pose Estimation](https://mlsanity.com/paper/2103.14843/) (2021)

## Built on by (most cited first, 25 of 25)

- [X-Pose: Detecting Any Keypoints](https://mlsanity.com/paper/2310.08530/) (2023)
- [LocLLM: Exploiting Generalizable Human Keypoint Localization via Large Language Model](https://mlsanity.com/paper/2406.04659/) (2024)
- [CapeX: Category-Agnostic Pose Estimation from Textual Point Explanation](https://mlsanity.com/paper/2406.00384/) (2024)
- [OpenKD: Opening Prompt Diversity for Zero- and Few-shot Keypoint Detection](https://mlsanity.com/paper/2409.19899/) (2024)
- [UniAP: Towards Universal Animal Perception in Vision via Few-shot Learning](https://mlsanity.com/paper/2308.09953/) (2023)
- [Jamais Vu: Exposing the Generalization Gap in Supervised Semantic Correspondence](https://mlsanity.com/paper/2506.08220/) (2025)
- [KptLLM: Unveiling the Power of Large Language Model for Keypoint Comprehension](https://mlsanity.com/paper/2411.01846/) (2024)
- [Subspace-Contrastive Multi-View Clustering](https://mlsanity.com/paper/2210.06795/) (2022)
- [PoseBench: Benchmarking the Robustness of Pose Estimation Models under Corruptions](https://mlsanity.com/paper/2406.14367/) (2024)
- [PartSeg: Few-shot Part Segmentation via Part-aware Prompt Learning](https://mlsanity.com/paper/2308.12757/) (2023)
- [Learning Structure-Supporting Dependencies via Keypoint Interactive Transformer for General Mammal Pose Estimation](https://mlsanity.com/paper/2502.18214/) (2025)
- [Probabilistic Prompt Distribution Learning for Animal Pose Estimation](https://mlsanity.com/paper/2503.16120/) (2025)
- [GEM: Context-Aware Gaze EstiMation with Visual Search Behavior Matching for Chest Radiograph](https://mlsanity.com/paper/2408.05502/) (2024)
- [ZebraPose: Zebra Detection and Pose Estimation using only Synthetic Data](https://mlsanity.com/paper/2408.10831/) (2024)
- [Towards Multi-Modal Animal Pose Estimation: A Survey and In-Depth Analysis](https://mlsanity.com/paper/2410.09312/) (2024)
- [Seamless Detection: Unifying Salient Object Detection and Camouflaged Object Detection](https://mlsanity.com/paper/2412.16840/) (2024)
- [Recurrent Feature Mining and Keypoint Mixup Padding for Category-Agnostic Pose Estimation](https://mlsanity.com/paper/2503.21140/) (2025)
- [MamKPD: A Simple Mamba Baseline for Real-Time 2D Keypoint Detection](https://mlsanity.com/paper/2412.01422/) (2024)
- [ActivityCLIP: Enhancing Group Activity Recognition by Mining Complementary Information from Text to Supplement Image Modality](https://mlsanity.com/paper/2407.19820/) (2024)
- [KptLLM++: Towards Generic Keypoint Comprehension with Large Language Model](https://mlsanity.com/paper/2507.11102/) (2025)
- [Parse Graph-Based Visual-Language Interaction for Human Pose Estimation](https://mlsanity.com/paper/2509.07385/) (2025)
- [Talking Points: Describing and Localizing Pixels](https://mlsanity.com/paper/2510.14583/) (2025)
- [Point-to-Point: Sparse Motion Guidance for Controllable Video Editing](https://mlsanity.com/paper/2511.18277/) (2025)
- [GenCape: Structure-Inductive Generative Modeling for Category-Agnostic Pose Estimation](https://mlsanity.com/paper/2605.13151/) (2026)
- [GKDT: General Keypoint Detection Transformer](https://mlsanity.com/paper/2607.00752/) (2026)

## Abstract (arXiv)

Animal pose estimation is challenging for existing image-based methods because of limited training data and large intra- and inter-species variances. Motivated by the progress of visual-language research, we propose that pre-trained language models (e.g., CLIP) can facilitate animal pose estimation by providing rich prior knowledge for describing animal keypoints in text. However, we found that building effective connections between pre-trained language models and visual animal keypoints is non-trivial since the gap between text-based descriptions and keypoint-based visual features about animal pose can be significant. To address this issue, we introduce a novel prompt-based Contrastive learning scheme for connecting Language and AniMal Pose (CLAMP) effectively. The CLAMP attempts to bridge the gap by adapting the text prompts to the animal keypoints during network training. The adaptation is decomposed into spatial-aware and feature-aware processes, and two novel contrastive losses are devised correspondingly. In practice, the CLAMP enables the first cross-modal animal pose estimation paradigm. Experimental results show that our method achieves state-of-the-art performance under the supervised, few-shot, and zero-shot settings, outperforming image-based methods by a large margin.