# A Light-Weight Contrastive Approach for Aligning Human Pose Sequences

arXiv: https://arxiv.org/abs/2303.04244 · HTML: https://mlsanity.com/paper/2303.04244/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 39 references that are in the map.
- First posted on arXiv 7 Mar 2023.

## Builds on (most cited first, 20 of 20)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [Perceiver: General Perception with Iterative Attention](https://mlsanity.com/paper/2103.03206/) (2021)
- [Revisiting Skeleton-based Action Recognition](https://mlsanity.com/paper/2104.13586/) (2021)
- [Temporal Cycle-Consistency Learning](https://mlsanity.com/paper/1904.07846/) (2019)
- [A New Representation of Skeleton Sequences for 3D Action Recognition](https://mlsanity.com/paper/1703.03492/) (2017)
- [BlazePose: On-device Real-time Body Pose tracking](https://mlsanity.com/paper/2006.10204/) (2020)
- [Learning by Aligning Videos in Time](https://mlsanity.com/paper/2103.17260/) (2021)
- [Representation Learning via Global Temporal Alignment and Cycle-Consistency](https://mlsanity.com/paper/2105.05217/) (2021)
- [SkeleMotion: A New Representation of Skeleton Joint Sequences Based on Motion Information for 3D Action Recognition](https://mlsanity.com/paper/1907.13025/) (2019)
- [MatConvNet - Convolutional Neural Networks for MATLAB](https://mlsanity.com/paper/1412.4564/) (2014)
- [SOMA: Solving Optical Marker-Based MoCap Automatically](https://mlsanity.com/paper/2110.04431/) (2021)
- [Learning to Align Sequential Actions in the Wild](https://mlsanity.com/paper/2111.09301/) (2021)
- [Context-Aware Sequence Alignment using 4D Skeletal Augmentation](https://mlsanity.com/paper/2204.12223/) (2022)
- [View-Invariant, Occlusion-Robust Probabilistic Embedding for Human Pose](https://mlsanity.com/paper/2010.13321/) (2020)

## Abstract (arXiv)

We present a simple unsupervised method for learning an encoder mapping short 3D pose sequences into embedding vectors suitable for sequence-to-sequence alignment by dynamic time warping. Training samples consist of temporal windows of frames containing 3D body points such as mocap markers or skeleton joints. A light-weight, 3-layer encoder is trained using a contrastive loss function that encourages embedding vectors of augmented sample pairs to have cosine similarity 1, and similarity 0 with all other samples in a minibatch. When multiple scripted training sequences are available, temporal alignments inferred from an initial round of training are harvested to extract additional, cross-performance match pairs for a second phase of training to refine the encoder. In addition to being simple, the proposed method is fast to train, making it easy to adapt to new data using different marker sets or skeletal joint layouts. Experimental results illustrate ease of use, transferability, and utility of the learned embeddings for comparing and analyzing human behavior sequences.