mlsanityData to 7 Oct 2026

Vision-Language Models for Ergonomic Assessment of Manual Lifting Tasks: Estimating Horizontal and Vertical Hand Distances from RGB Video

arXiv 2602.20658 ↗First posted 24 Feb 2026cs.CV cs.AI cs.HCcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Manual lifting tasks are a major contributor to work-related musculoskeletal disorders, and effective ergonomic risk assessment is essential for quantifying physical exposure and informing ergonomic interventions. The Revised NIOSH Lifting Equation (RNLE) is a widely used ergonomic risk assessment tool for lifting tasks that relies on six task variables, including horizontal (H) and vertical (V) hand distances; such distances are typically obtained through manual measurement or specialized sensing systems and are difficult to use in real-world environments. We evaluated the feasibility of using innovative vision-language models (VLMs) to non-invasively estimate H and V from RGB video streams. Two multi-stage VLM-based pipelines were developed: a text-guided detection-only pipeline and a detection-plus-segmentation pipeline. Both pipelines used text-guided localization of task-relevant regions of interest, visual feature extraction from those regions, and transformer-based temporal regression to estimate H and V at the start and end of a lift. For a range of lifting tasks, estimation performance was evaluated using leave-one-subject-out validation across the two pipelines and seven camera view conditions. Results varied significantly across pipelines and camera view conditions, with the segmentation-based, multi-view pipeline consistently yielding the smallest errors, achieving mean absolute errors of approximately 6-8 cm when estimating H and 5-8 cm when estimating V. Across pipelines and camera view configurations, pixel-level segmentation reduced estimation error by approximately 20-30% for H and 35-40% for V relative to the detection-only pipeline. These findings support the feasibility of VLM-based pipelines for video-based estimation of RNLE distance parameters.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 14 of its 40 references that are in the map. First posted on arXiv 24 Feb 2026.

Builds on

14 of 40 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Do Vision-Language Models Represent Space and How? Evaluating Spatial Frame of Reference Under Ambiguities22 Oct
  2. 2024Reasoning Paths with Reference Objects Elicit Quantitative Spatial Reasoning in Large Vision-Language Models15 Sep

Other references · 12

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  5. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
  6. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  7. 2019Cross View Fusion for 3D Human Pose Estimationcited by 90
  8. 2021Multi-View Depth Estimation by Fusing Single-View Depth Probability with Multi-View Geometrycited by 46
  9. 2023Boosting Segment Anything Model Towards Open-Vocabulary Learningcited by 10
  10. 2020Temporal Smoothing for 3D Human Pose Estimation and Localization for Occluded Peoplecited by 8
  11. 2024ErgoChat: a Visual Query System for the Ergonomic Risk Assessment of Construction Workerscited by 6
  12. 2025RoHOI: Robustness Benchmark for Human-Object Interaction Detectioncited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Vision-Language Models for Occupational Physical Exposure Assessment: Estimating External Hand Forces in Manual Material Handling Tasks from RGB Video
  2. 2025The Potential and Limitations of Vision-Language Models for Human Motion Understanding: A Case Study in Data-Driven Stroke Rehabilitation
  3. 2025Using Visual Language Models to Control Bionic Hands: Assessment of Object Perception and Grasp Inference
  4. 2023Online Action Recognition for Human Risk Prediction with Anticipated Haptic Alert via Wearables
  5. 2026Can Vision Language Models Judge Action Quality? An Empirical Evaluation
  6. 2025Reducing Annotation Burden in Physical Activity Research Using Vision-Language Models
  7. 2026HandVQA: Diagnosing and Improving Fine-Grained Spatial Reasoning about Hands in Vision-Language Models
  8. 2025Parallels Between VLA Model Post-Training and Human Motor Learning: Progress, Challenges, and Trends