mlsanityData to 7 Oct 2026

Capek 0.5: An Execution-Centric Vision-Language Model for Embodied Intelligence

arXiv 2608.06756 ↗First posted 7 Aug 2026cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Vision-language models are increasingly serving as the reasoning core of embodied agents. Robot execution is inherently iterative: each action reshapes the scene and physical state, continually renewing what must be perceived, reasoned about, and verified. Meeting these demands requires complementary capabilities that differ in supervision signals, prediction formats, and verification criteria. Existing approaches typically develop these capabilities against isolated, task-specific objectives, leaving open how they should be organized and integrated around execution as a whole. We present Capek 0.5, an embodied vision-language model built around an execution-centric capability taxonomy. Rather than organizing training by datasets or tasks, the taxonomy groups embodied capabilities according to their functional roles throughout execution and comprises four capability families: Spatial Reasoning, Temporal Understanding, Action Guidance, and State Verification. Each capability is first acquired by a dedicated specialist through reinforcement learning with verifiable rewards from a shared backbone, and the specialists are then consolidated into a single inference-time model through weight-space merging followed by routed policy-space distillation. We instantiate Capek 0.5 at the 2B and 35B-A3B scales and evaluate it from three complementary perspectives: comprehensive benchmark suites including Capek-StateBench, a new benchmark for state verification; a controlled study of capability retention from specialists to the unified model; and closed-loop evaluation in simulated embodied environments. Capek 0.5 improves the large majority of matched benchmark rows over its initialization, retains all four specialized capabilities in one checkpoint with quantified losses, and transfers to closed-loop embodied task execution.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 83 of its 99 references that are in the map. First posted on arXiv 7 Aug 2026.

Builds on

83 of 99 references.

Closest ancestors · 16

Same direction, newest first

  1. 2025NaviTrace: Evaluating Embodied Navigation of Vision-Language Models30 Oct
  2. 2025Point-It-Out: Benchmarking Embodied Reasoning for Vision Language Models in Multi-Stage Visual Grounding30 Sep
  3. 2025Towards Embodied Agentic AI: Review and Classification of LLM- and VLM-Driven Robot Autonomy and Interaction7 Aug
  4. 2025MindCube: Spatial Mental Modeling from Limited Views26 Jun
  5. 2025OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Models3 Jun
  6. 2025Robo2VLM: Visual Question Answering from Large-Scale In-the-Wild Robot Manipulation Datasets21 May
  7. 2025From Seeing to Doing: Bridging Reasoning and Decision for Robotic Manipulation13 May
  8. 2025From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3D29 Mar
  9. 2025Omnia de EgoTempo: Benchmarking Temporal Understanding of Multi-Modal LLMs in Egocentric Videos17 Mar
  10. 2025RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concrete28 Feb
  11. 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents13 Feb
  12. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces18 Dec
4 more
  1. 2024RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics25 Nov
  2. 2024RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotics15 Jun
  3. 2024EmbSpatial-Bench: Benchmarking Spatial Understanding for Embodied Tasks with Large Vision-Language Models9 Jun
  4. 2023PaLM-E: An Embodied Multimodal Language Model6 Mar

Other references · 67

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2023Visual Instruction Tuningcited by 7,792
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  6. 2025Qwen2.5-VL Technical Reportcited by 5,418
  7. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  8. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  9. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  10. 2025Qwen3-VL Technical Reportcited by 2,534
  11. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  12. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
38 more
  1. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  2. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  3. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  4. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  5. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  6. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
  7. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  8. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  9. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  10. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  11. 2022Editing Models with Task Arithmeticcited by 1,076
  12. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  13. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 840
  14. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  15. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
  16. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 795
  17. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 735
  18. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  19. 2017Localizing Moments in Video with Natural Languagecited by 660
  20. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 644
  21. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
  22. 2016Hollywood in Homes: Crowdsourcing Data Collection for Activity Understandingcited by 535
  23. 2019CLEVRER: CoLlision Events for Video REpresentation and Reasoningcited by 475
  24. 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 390
  25. 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 386
  26. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
  27. 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 321
  28. 2023Perception Test: A Diagnostic Benchmark for Multimodal Video Modelscited by 310
  29. 2024STAR: A Benchmark for Situated Reasoning in Real-World Videoscited by 213
  30. 2025Cosmos-Reason1: From Physical Common Sense To Embodied Reasoningcited by 159
  31. 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 137
  32. 2023RoboVQA: Multimodal Long-Horizon Reasoning for Roboticscited by 116
  33. 2025Gemini Robotics 1.5: Pushing the Frontier of Generalist Robots with Advanced Embodied Reasoning, Thinking, and Motion Transfercited by 86
  34. 2023Hierarchical Video-Moment Retrieval and Step-Captioningcited by 83
  35. 2025Scaling RL to Long Videoscited by 83
  36. 2025Scaling Spatial Intelligence with Multimodal Foundation Modelscited by 75
  37. 2024VideoEspresso: A Large-Scale Chain-of-Thought Dataset for Fine-Grained Video Reasoning via Core Frame Selectioncited by 64
  38. 2026MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Trainingcited by 60

50 most cited of 67 All 67 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Unmasking the Illusion of Embodied Reasoning in Vision-Language-Action Models
  2. 2026GigaBrain-0.7: Scaling Embodied Foundation Models to Emergent Capabilities with a Three-System Architecture
  3. 2025Embodied4C: Measuring What Matters for Embodied Vision-Language Navigation
  4. 2026RoboAgent: Chaining Basic Capabilities for Embodied Task Planning
  5. 2026From Inference Efficiency to Embodied Efficiency: Revisiting Efficiency Metrics for Vision-Language-Action Models
  6. 2026HiMe: Hierarchical Embodied Memory for Long-Horizon Vision-Language-Action Control
  7. 2025EmbodiedBrain: Expanding Performance Boundaries of Task Planning for Embodied Intelligence
  8. 2026Vision-Language-Action in Robotics: A Survey of Datasets, Benchmarks, and Data Engines