# SKI Models: Skeleton Induced Vision-Language Embeddings for Understanding Activities of Daily Living

arXiv: https://arxiv.org/abs/2502.03459 · HTML: https://mlsanity.com/paper/2502.03459/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models.
- First posted on arXiv 5 Feb 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Video understanding with MLLMs](https://mlsanity.com/mllm/video-understanding-with-mllms/)

## Built on by (most cited first, 3 of 3)

- [Universal Skeleton Understanding via Differentiable Rendering and MLLMs](https://mlsanity.com/paper/2603.18003/) (2026)
- [Superman: Unifying Skeleton and Vision for Human Motion Perception and Generation](https://mlsanity.com/paper/2602.02401/) (2026)
- [PoseBridge: Bridging the Skeletonization Gap for Zero-Shot Skeleton-Based Action Recognition](https://mlsanity.com/paper/2605.11497/) (2026)

## Abstract (arXiv)

The introduction of vision-language models like CLIP has enabled the development of foundational video models capable of generalizing to unseen videos and human actions. However, these models are typically trained on web videos, which often fail to capture the challenges present in Activities of Daily Living (ADL) videos. Existing works address ADL-specific challenges, such as similar appearances, subtle motion patterns, and multiple viewpoints, by combining 3D skeletons and RGB videos. However, these approaches are not integrated with language, limiting their ability to generalize to unseen action classes. In this paper, we introduce SKI models, which integrate 3D skeletons into the vision-language embedding space. SKI models leverage a skeleton-language model, SkeletonCLIP, to infuse skeleton information into Vision Language Models (VLMs) and Large Vision Language Models (LVLMs) through collaborative training. Notably, SKI models do not require skeleton data during inference, enhancing their robustness for real-world applications. The effectiveness of SKI models is validated on three popular ADL datasets for zero-shot action recognition and video caption generation tasks.