# Identity Preserve Transform: Understand What Activity Classification Models Have Learnt

arXiv: https://arxiv.org/abs/1912.06314 · HTML: https://mlsanity.com/paper/1912.06314/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Dec 2019.

## Built on by (most cited first, 2 of 2)

- [Fine-grained Action Analysis: A Multi-modality and Multi-task Dataset of Figure Skating](https://mlsanity.com/paper/2307.02730/) (2023)
- [Nuisance-Label Supervision: Robustness Improvement by Free Labels](https://mlsanity.com/paper/2110.07118/) (2021)

## Abstract (arXiv)

Activity classification has observed great success recently. The performance on small dataset is almost saturated and people are moving towards larger datasets. What leads to the performance gain on the model and what the model has learnt? In this paper we propose identity preserve transform (IPT) to study this problem. IPT manipulates the nuisance factors (background, viewpoint, etc.) of the data while keeping those factors related to the task (human motion) unchanged. To our surprise, we found popular models are using highly correlated information (background, object) to achieve high classification accuracy, rather than using the essential information (human motion). This can explain why an activity classification model usually fails to generalize to datasets it is not trained on. We implement IPT in two forms, i.e. image-space transform and 3D transform, using synthetic images. The tool will be made open-source to help study model and dataset design.