mlsanityData to 13 Oct 2026

AutoComPose: Automatic Generation of Pose Transition Descriptions for Composed Pose Retrieval Using Multimodal LLMs

arXiv 2503.22884 ↗First posted 28 Mar 2025cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

Abstract (arXiv)

Composed pose retrieval (CPR) enables users to search for human poses by specifying a reference pose and a transition description, but progress in this field is hindered by the scarcity and inconsistency of annotated pose transitions. Existing CPR datasets rely on costly human annotations or heuristic-based rule generation, both of which limit scalability and diversity. In this work, we introduce AutoComPose, the first framework that leverages multimodal large language models (MLLMs) to automatically generate rich and structured pose transition descriptions. Our method enhances annotation quality by structuring transitions into fine-grained body part movements and introducing mirrored/swapped variations, while a cyclic consistency constraint ensures logical coherence between forward and reverse transitions. To advance CPR research, we construct and release two dedicated benchmarks, AIST-CPR and PoseFixCPR, supplementing prior datasets with enhanced attributes. Extensive experiments demonstrate that training retrieval models with AutoComPose yields superior performance over human-annotated and heuristic-based methods, significantly reducing annotation costs while improving retrieval quality. Our work pioneers the automatic annotation of pose transitions, establishing a scalable foundation for future CPR research.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 38 of its 56 references that are in the map. First posted on arXiv 28 Mar 2025.

Builds on

38 of 56 references.

References · 38

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
  2. 2024GPT-4o System Cardcited by 4,536
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,389
  4. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,314
  5. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,603
  6. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,213
  7. 2017CyCADA: Cycle-Consistent Adversarial Domain Adaptationcited by 1,079
  8. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,062
  9. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 613
  10. 2018DensePose: Dense Human Pose Estimation In The Wildcited by 573
  11. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 498
  12. 2021AI Choreographer: Music Conditioned 3D Dance Generation with AIST++cited by 481
26 more
  1. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 384
  2. 2021Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Modelscited by 249
  3. 2022Selective Annotation Makes Language Models Better Few-Shot Learnerscited by 243
  4. 2018Composing Text and Image for Image Retrieval - An Empirical Odysseycited by 236
  5. 2023BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motioncited by 232
  6. 2021AGORA: Avatars in Geography Optimized for Regression Analysiscited by 204
  7. 2024Large Language Models for Data Annotation and Synthesis: A Surveycited by 204
  8. 2023Multimodal Large Language Models: A Surveycited by 194
  9. 2023Zero-Shot Composed Image Retrieval with Textual Inversioncited by 174
  10. 2023Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrievalcited by 156
  11. 2024VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Taskscited by 136
  12. 2018Ordinal Depth Supervision for 3D Human Pose Estimationcited by 107
  13. 2023ChatPose: Chatting about 3D Human Posecited by 71
  14. 2017Multi-Context Attention for Human Pose Estimationcited by 66
  15. 2023Composed Image Retrieval using Contrastive Learning and Task-oriented CLIP-based Featurescited by 52
  16. 2024Efficient Large Multi-modal Models via Visual Context Compressioncited by 46
  17. 2023PoseFix: Correcting 3D Human Poses with Natural Languagecited by 39
  18. 2016MoCap-guided Data Augmentation for 3D Pose Estimation in the Wildcited by 37
  19. 2024Simple but Effective Raw-Data Level Multimodal Fusion for Composed Image Retrievalcited by 24
  20. 2025A Comprehensive Survey on Composed Image Retrievalcited by 17
  21. 2024Visual Delta Generator with Large Multi-modal Models for Semi-supervised Composed Image Retrievalcited by 13
  22. 2021FixMyPose: Pose Correctional Captioning and Retrievalcited by 12
  23. 2019Linking Art through Human Posescited by 9
  24. 2023NEUCORE: Neural Concept Reasoning for Composed Image Retrievalcited by 8
  25. 2024Diversifying Human Pose in Synthetic Data for Aerial-view Human Detectioncited by 5
  26. 2015Pose Embeddings: A Deep Architecture for Learning to Match Human Posescited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024CapeLLM: Support-Free Category-Agnostic Pose Estimation with Multimodal Large Language Models
  2. 2024Pose Priors from Language Models
  3. 2025LLaVA-Pose: Enhancing Human Pose and Action Understanding via Keypoint-Integrated Instruction Tuning
  4. 2025IDMR: Towards Instance-Driven Precise Visual Correspondence in Multimodal Retrieval
  5. 2025HMVLM: Human Motion-Vision-Lanuage Model via MoE LoRA
  6. 2025Revisiting Reliability in the Reasoning-based Pose Estimation Benchmark
  7. 2024MotionGPT-2: A General-Purpose Motion-Language Model for Motion Generation and Understanding
  8. 2025PoseLLM: Enhancing Language-Guided Human Pose Estimation with MLP Alignment