# Test-Time Personalization with a Transformer for Human Pose Estimation

arXiv: https://arxiv.org/abs/2107.02133 · HTML: https://mlsanity.com/paper/2107.02133/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 62 of its 74 references that are in the map.
- First posted on arXiv 5 Jul 2021.

## Builds on (most cited first, 30 of 62)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [Object-Centric Learning with Slot Attention](https://mlsanity.com/paper/2006.15055/) (2020)
- [Image Transformer](https://mlsanity.com/paper/1802.05751/) (2018)
- [VideoBERT: A Joint Model for Video and Language Representation Learning](https://mlsanity.com/paper/1904.01766/) (2019)
- [Stand-Alone Self-Attention in Vision Models](https://mlsanity.com/paper/1906.05909/) (2019)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [3D human pose estimation in video with temporal convolutions and semi-supervised training](https://mlsanity.com/paper/1811.11742/) (2018)
- [Cascaded Pyramid Network for Multi-Person Pose Estimation](https://mlsanity.com/paper/1711.07319/) (2017)
- [GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond](https://mlsanity.com/paper/1904.11492/) (2019)
- [Convolutional Pose Machines](https://mlsanity.com/paper/1602.00134/) (2016)
- [Exploring Self-attention for Image Recognition](https://mlsanity.com/paper/2004.13621/) (2020)
- [DeepPose: Human Pose Estimation via Deep Neural Networks](https://mlsanity.com/paper/1312.4659/) (2013)
- [Relation Networks for Object Detection](https://mlsanity.com/paper/1711.11575/) (2017)
- ["Zero-Shot" Super-Resolution using Deep Internal Learning](https://mlsanity.com/paper/1712.06087/) (2017)
- [HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation](https://mlsanity.com/paper/1908.10357/) (2019)
- [Semantic Photo Manipulation with a Generative Image Prior](https://mlsanity.com/paper/2005.07727/) (2020)

## Built on by (most cited first, 30 of 33)

- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)
- [MM-TTA: Multi-Modal Test-Time Adaptation for 3D Semantic Segmentation](https://mlsanity.com/paper/2204.12667/) (2022)
- [MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction](https://mlsanity.com/paper/2204.07908/) (2022)
- [Coarse-to-Fine Sparse Transformer for Hyperspectral Image Reconstruction](https://mlsanity.com/paper/2203.04845/) (2022)
- [Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging](https://mlsanity.com/paper/2205.10102/) (2022)
- [On Pitfalls of Test-Time Adaptation](https://mlsanity.com/paper/2306.03536/) (2023)
- [Meta-DMoE: Adapting to Domain Shift by Meta-Distillation from Mixture-of-Experts](https://mlsanity.com/paper/2210.03885/) (2022)
- [Improved Test-Time Adaptation for Domain Generalization](https://mlsanity.com/paper/2304.04494/) (2023)
- [Towards Robust and Adaptive Motion Forecasting: A Causal Representation Perspective](https://mlsanity.com/paper/2111.14820/) (2021)
- [Dual Prototype Evolving for Test-Time Generalization of Vision-Language Models](https://mlsanity.com/paper/2410.12790/) (2024)
- [Recurrence without Recurrence: Stable Video Landmark Detection with Deep Equilibrium Models](https://mlsanity.com/paper/2304.00600/) (2023)
- [Neuro-Modulated Hebbian Learning for Fully Test-Time Adaptation](https://mlsanity.com/paper/2303.00914/) (2023)
- [AUTO: Adaptive Outlier Optimization for Test-Time OOD Detection](https://mlsanity.com/paper/2303.12267/) (2023)
- [ForgeryTTT: Zero-Shot Image Manipulation Localization with Test-Time Training](https://mlsanity.com/paper/2410.04032/) (2024)
- [Boost Test-Time Performance with Closed-Loop Inference](https://mlsanity.com/paper/2203.10853/) (2022)
- [Self-Correctable and Adaptable Inference for Generalizable Human Pose Estimation](https://mlsanity.com/paper/2303.11180/) (2023)
- [Improving ProtoNet for Few-Shot Video Object Recognition: Winner of ORBIT Challenge 2022](https://mlsanity.com/paper/2210.00174/) (2022)
- [From Question to Exploration: Test-Time Adaptation in Semantic Segmentation?](https://mlsanity.com/paper/2310.05341/) (2023)
- [Skeleton2Humanoid: Animating Simulated Characters for Physically-plausible Motion In-betweening](https://mlsanity.com/paper/2210.04294/) (2022)
- [Active Transfer Learning for Efficient Video-Specific Human Pose Estimation](https://mlsanity.com/paper/2311.05041/) (2023)
- [Align the GAP: Prior-based Unified Multi-Task Remote Physiological Measurement Framework For Domain Generalization and Personalization](https://mlsanity.com/paper/2506.16160/) (2025)
- [PointMAC: Meta-Learned Adaptation for Robust Test-Time Point Cloud Completion](https://mlsanity.com/paper/2510.10365/) (2025)
- [Generalized Robust Test-Time Adaptation in Continuous Dynamic Scenarios](https://mlsanity.com/paper/2310.04714/) (2023)
- [Not Only Consistency: Enhance Test-Time Adaptation with Spatio-temporal Inconsistency for Remote Physiological Measurement](https://mlsanity.com/paper/2507.07908/) (2025)
- [Towards Egocentric 3D Hand Pose Estimation in Unseen Domains](https://mlsanity.com/paper/2601.06537/) (2026)
- [Meta-Auxiliary Learning for Adaptive Human Pose Prediction](https://mlsanity.com/paper/2304.06411/) (2023)
- [Boost Video Frame Interpolation via Motion Adaptation](https://mlsanity.com/paper/2306.13933/) (2023)
- [H-InDex: Visual Reinforcement Learning with Hand-Informed Representations for Dexterous Manipulation](https://mlsanity.com/paper/2310.01404/) (2023)
- [Test-Time Adaptation for Keypoint-Based Spacecraft Pose Estimation Based on Predicted-View Synthesis](https://mlsanity.com/paper/2410.04298/) (2024)
- [Differentially Private 2D Human Pose Estimation](https://mlsanity.com/paper/2504.10190/) (2025)

## Abstract (arXiv)

We propose to personalize a human pose estimator given a set of test images of a person without using any manual annotations. While there is a significant advancement in human pose estimation, it is still very challenging for a model to generalize to different unknown environments and unseen persons. Instead of using a fixed model for every test case, we adapt our pose estimator during test time to exploit person-specific information. We first train our model on diverse data with both a supervised and a self-supervised pose estimation objectives jointly. We use a Transformer model to build a transformation between the self-supervised keypoints and the supervised keypoints. During test time, we personalize and adapt our model by fine-tuning with the self-supervised objective. The pose is then improved by transforming the updated self-supervised keypoints. We experiment with multiple datasets and show significant improvements on pose estimations with our self-supervised personalization.