# Temporal Visual Semantics-Induced Human Motion Understanding with Large Language Models

arXiv: https://arxiv.org/abs/2512.22249 · HTML: https://mlsanity.com/paper/2512.22249/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 118 references that are in the map.
- First posted on arXiv 24 Dec 2025.

## Builds on (most cited first, 29 of 29)

- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understanding](https://mlsanity.com/paper/2412.00493/) (2024)
- [ST-LLM: Large Language Models Are Effective Temporal Learners](https://mlsanity.com/paper/2404.00308/) (2024)
- [From Images to Textual Prompts: Zero-shot VQA with Frozen Large Language Models](https://mlsanity.com/paper/2212.10846/) (2022)
- [MotionLLM: Understanding Human Behaviors from Human Motions and Videos](https://mlsanity.com/paper/2405.20340/) (2024)
- [Segmental Spatiotemporal CNNs for Fine-grained Action Segmentation](https://mlsanity.com/paper/1602.02995/) (2016)
- [VideoRefer Suite: Advancing Spatial-Temporal Object Understanding with Video LLM](https://mlsanity.com/paper/2501.00599/) (2024)
- [LLMs as Zero-shot Graph Learners: Alignment of GNN Representations with LLM Token Embeddings](https://mlsanity.com/paper/2408.14512/) (2024)
- [SensorLLM: Aligning Large Language Models with Motion Sensors for Human Activity Recognition](https://mlsanity.com/paper/2410.10624/) (2024)
- [LSceneLLM: Enhancing Large 3D Scene Understanding Using Adaptive Visual Preferences](https://mlsanity.com/paper/2412.01292/) (2024)
- [Human Motion Instruction Tuning](https://mlsanity.com/paper/2411.16805/) (2024)
- [Visual Fact Checker: Enabling High-Fidelity Detailed Caption Generation](https://mlsanity.com/paper/2404.19752/) (2024)
- [Scaling Large Motion Models with Million-Level Human Motions](https://mlsanity.com/paper/2410.03311/) (2024)
- [Breaking Down Video LLM Benchmarks: Knowledge, Spatial Perception, or True Temporal Understanding?](https://mlsanity.com/paper/2505.14321/) (2025)
- [Self-Supervised Information Bottleneck for Deep Multi-View Subspace Clustering](https://mlsanity.com/paper/2204.12496/) (2022)
- [Research on reinforcement learning based warehouse robot navigation algorithm in complex warehouse layout](https://mlsanity.com/paper/2411.06128/) (2024)
- [Do Language Models Understand Time?](https://mlsanity.com/paper/2412.13845/) (2024)
- [Seq2Time: Sequential Knowledge Transfer for Video LLM Temporal Grounding](https://mlsanity.com/paper/2411.16932/) (2024)
- [Multimodal Large Models Are Effective Action Anticipators](https://mlsanity.com/paper/2501.00795/) (2025)
- [Consistency and Diversity induced Human Motion Segmentation](https://mlsanity.com/paper/2202.04861/) (2022)
- [Constructing Indoor Region-based Radio Map without Location Labels](https://mlsanity.com/paper/2308.16759/) (2023)
- [Graph Constrained Data Representation Learning for Human Motion Segmentation](https://mlsanity.com/paper/2107.13362/) (2021)
- [PointLIE: Locally Invertible Embedding for Point Cloud Sampling and Recovery](https://mlsanity.com/paper/2104.14769/) (2021)
- [Blind Construction of Angular Power Maps in Massive MIMO Networks](https://mlsanity.com/paper/2510.07071/) (2025)
- [Block-Diagonal Guided DBSCAN Clustering](https://mlsanity.com/paper/2404.01341/) (2024)
- [Trajectory Map-Matching in Urban Road Networks Based on RSS Measurements](https://mlsanity.com/paper/2502.01280/) (2025)
- [Credit Risk Analysis for SMEs Using Graph Neural Networks in Supply Chain](https://mlsanity.com/paper/2507.07854/) (2025)
- [Unsupervised Radio Map Construction in Mixed LoS/NLoS Indoor Environments](https://mlsanity.com/paper/2510.08015/) (2025)
- [Blind Radio Mapping via Spatially Regularized Bayesian Trajectory Inference](https://mlsanity.com/paper/2512.13701/) (2025)

## Built on by (most cited first, 1 of 1)

- [D-CLOT: Double Closed Loop Optimal Transport for Unsupervised Action Segmentation](https://mlsanity.com/paper/2608.05877/) (2026)

## Abstract (arXiv)

Unsupervised human motion segmentation (HMS) can be effectively achieved using subspace clustering techniques. However, traditional methods overlook the role of temporal semantic exploration in HMS. This paper explores the use of temporal vision semantics (TVS) derived from human motion sequences, leveraging the image-to-text capabilities of a large language model (LLM) to enhance subspace clustering performance. The core idea is to extract textual motion information from consecutive frames via LLM and incorporate this learned information into the subspace clustering framework. The primary challenge lies in learning TVS from human motion sequences using LLM and integrating this information into subspace clustering. To address this, we determine whether consecutive frames depict the same motion by querying the LLM and subsequently learn temporal neighboring information based on its response. We then develop a TVS-integrated subspace clustering approach, incorporating subspace embedding with a temporal regularizer that induces each frame to share similar subspace embeddings with its temporal neighbors. Additionally, segmentation is performed based on subspace embedding with a temporal constraint that induces the grouping of each frame with its temporal neighbors. We also introduce a feedback-enabled framework that continuously optimizes subspace embedding based on the segmentation output. Experimental results demonstrate that the proposed method outperforms existing state-of-the-art approaches on four benchmark human motion datasets.