mlsanityData to 14 Oct 2026

An Architecture-Led Hybrid Report on Body Language Detection Project

arXiv 2512.23028 ↗First posted 28 Dec 2025cs.CV cs.AI cs.SEcited by 0

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

Abstract (arXiv)

This report provides an architecture-led analysis of two modern vision-language models (VLMs), Qwen2.5-VL-7B-Instruct and Llama-4-Scout-17B-16E-Instruct, and explains how their architectural properties map to a practical video-to-artifact pipeline implemented in the BodyLanguageDetection repository [1]. The system samples video frames, prompts a VLM to detect visible people and generate pixel-space bounding boxes with prompt-conditioned attributes (emotion by default), validates output structure using a predefined schema, and optionally renders an annotated video. We first summarize the shared multimodal foundation (visual tokenization, Transformer attention, and instruction following), then describe each architecture at a level sufficient to justify engineering choices without speculative internals. Finally, we connect model behavior to system constraints: structured outputs can be syntactically valid while semantically incorrect, schema validation is structural (not geometric correctness), person identifiers are frame-local in the current prompting contract, and interactive single-frame analysis returns free-form text rather than schema-enforced JSON. These distinctions are critical for writing defensible claims, designing robust interfaces, and planning evaluation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 6 of its 17 references that are in the map. First posted on arXiv 28 Dec 2025.

Builds on

6 of 17 references.

References · 6

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  5. 2024The Butterfly Effect of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model Performancecited by 100
  6. 2021Ethics Sheet for Automatic Emotion Recognition and Sentiment Analysiscited by 22

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Can Vision-Language Models Analyze Human-Centered Video? Mapping Model Capabilities and Human-AI Collaborative Workflows
  2. 2024MVP-Bench: Can Large Vision--Language Models Conduct Multi-level Visual Perception Like Humans?
  3. 2026FineBench: Benchmarking and Enhancing Vision-Language Models for Fine-grained Human Activity Understanding
  4. 2024Contextual Emotion Recognition using Large Vision Language Models
  5. 2024HumanVBench: Probing Human-Centric Video Understanding in MLLMs with Automatically Synthesized Benchmarks
  6. 2025Customizing Visual Emotion Evaluation for MLLMs: An Open-vocabulary, Multifaceted, and Scalable Approach
  7. 2025HumanVideo-MME: Benchmarking MLLMs for Human-Centric Video Understanding
  8. 2025V2P-Bench: Evaluating Video-Language Understanding with Visual Prompts for Better Human-Model Interaction