mlsanityData to 14 Oct 2026

Human-Centric Image Captioning with Subject-Centered Spatial Understanding

arXiv 2609.08300 ↗First posted 8 Sep 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

While multimodal large language models (MLLMs) achieve remarkable performance on generic image captioning, they frequently suffer from structural hallucinations in human-centric scenarios. Accurately modeling human subjects is foundational for critical downstream applications, such as accurate avatar/video/image generation and fine-grained human action understanding. However, these tasks require highly precise subject-centered spatial grounding, such as distinguishing egocentric left/right laterality and maintaining correct anatomical-object bindings. Although catastrophic for structural integrity, these localized spatial inversions are often overshadowed by overall descriptive metrics in existing benchmarks. To systematically expose and quantify this bottleneck, we introduce SPACE (Subject-centric Poses, Appearance, and Characteristics Evaluation), a benchmark designed to evaluate subject-centered spatial understanding. On SPACE, we reveal that despite strong generic perception, current MLLMs consistently fail to ground descriptions in the subject's intrinsic frame of reference. To bridge this gap, we propose a specialized data construction and alignment pipeline. We first extract structured spatial hints from fine-grained body-part localization to guide a two-stage caption rewriting process, yielding highly spatially-faithful training data. Furthermore, we design a rubric-based reward for Group Relative Policy Optimization (GRPO) that explicitly penalizes structurally critical spatial errors during alignment. Extensive experiments on SPACE demonstrate our framework significantly improves human-centric caption quality, particularly in subject-centered spatial reasoning, achieving performance competitive with strong closed-source models. Our benchmark and code are available at https://github.com/JHang2020/SPACE-Eval.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 35 of its 59 references that are in the map. First posted on arXiv 8 Sep 2026.

Builds on

35 of 59 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Views21 Oct
  2. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning1 Jul
  3. 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligence29 May
  4. 2025VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction26 May
  5. 2025InSpire: Vision-Language-Action Models with Intrinsic Spatial Reasoning20 May
  6. 2025SpaceR: Reinforcing MLLMs in Video Spatial Reasoning2 Apr
  7. 2024SAT: Dynamic Spatial Aptitude Training for Multimodal Language Models10 Dec
  8. 2024Do Vision-Language Models Represent Space and How? Evaluating Spatial Frame of Reference Under Ambiguities22 Oct

Other references · 27

  1. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  2. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  3. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  4. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 735
  5. 2026Kimi K2.5: Visual Agentic Intelligencecited by 501
  6. 2024YOLOv11: An Overview of the Key Architectural Enhancementscited by 425
  7. 2026Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Rankingcited by 230
  8. 2025MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipecited by 186
  9. 2025Cambrian-S: Towards Spatial Supersensing in Videocited by 134
  10. 2024Prometheus-Vision: Vision-Language Model as a Judge for Fine-Grained Evaluationcited by 125
  11. 2023ChatGPT Asks, BLIP-2 Answers: Automatic Questioning Towards Enriched Visual Descriptionscited by 86
  12. 2024A Survey of Multimodal Large Language Model from A Data-centric Perspectivecited by 75
15 more
  1. 2025SpatialReasoner: Towards Explicit and Generalizable 3D Spatial Reasoningcited by 68
  2. 2024ImageInWords: Unlocking Hyper-Detailed Image Descriptionscited by 57
  3. 2025Chasing the Tail: Effective Rubric-based Reward Modeling for Large Language Model Post-Trainingcited by 53
  4. 2024Harnessing the Power of MLLMs for Transferable Text-to-Image Person ReIDcited by 39
  5. 2024Task Preference Optimization: Improving Multimodal Large Language Models with Vision Task Alignmentcited by 29
  6. 2026Open Rubric System: Scaling Reinforcement Learning with Pairwise Adaptive Rubriccited by 27
  7. 2025CAPability: A Comprehensive Visual Caption Benchmark for Evaluating Both Correctness and Thoroughnesscited by 25
  8. 2025R3: Robust Rubric-Agnostic Reward Modelscited by 23
  9. 2025T5Gemma 2: Seeing, Reading, and Understanding Longercited by 17
  10. 2025FoREST: Frame of Reference Evaluation in Spatial Reasoning Taskscited by 9
  11. 2025The Unseen Bias: How Norm Discrepancy in Pre-Norm MLLMs Leads to Visual Information Losscited by 5
  12. 2023HL Dataset: Visually-grounded Description of Scenes, Actions and Rationalescited by 4
  13. 2026Semantic Routing: Exploring Multi-Layer LLM Feature Weighting for Diffusion Transformerscited by 4
  14. 2025ID-Align: RoPE-Conscious Position Remapping for Dynamic High-Resolution Adaptation in Vision-Language Modelscited by 2
  15. 2025GRAN-TED: Generating Robust, Aligned, and Nuanced Text Embedding for Diffusion Modelscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025C2F-Space: Coarse-to-Fine Space Grounding for Spatial Instructions using Vision-Language Models
  2. 2024A dual contrastive framework
  3. 20243D Spatial Understanding in MLLMs: Disambiguation and Evaluation
  4. 2025Spatial-LLaVA: Enhancing Large Language Models with Spatial Referring Expressions for Visual Understanding
  5. 2025Human-MME: A Holistic Evaluation Benchmark for Human-Centric Multimodal Large Language Models
  6. 2026GAVEL: Grounded Caption Error Verification and Localization
  7. 2026From Correspondence to Actions: Human-Like Multi-Image Spatial Reasoning in Multi-modal Large Language Models
  8. 2024HumanVLM: Foundation for Human-Scene Vision-Language Model