Capek 0.5: An Execution-Centric Vision-Language Model for Embodied Intelligence
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
Abstract (arXiv)
Vision-language models are increasingly serving as the reasoning core of embodied agents. Robot execution is inherently iterative: each action reshapes the scene and physical state, continually renewing what must be perceived, reasoned about, and verified. Meeting these demands requires complementary capabilities that differ in supervision signals, prediction formats, and verification criteria. Existing approaches typically develop these capabilities against isolated, task-specific objectives, leaving open how they should be organized and integrated around execution as a whole. We present Capek 0.5, an embodied vision-language model built around an execution-centric capability taxonomy. Rather than organizing training by datasets or tasks, the taxonomy groups embodied capabilities according to their functional roles throughout execution and comprises four capability families: Spatial Reasoning, Temporal Understanding, Action Guidance, and State Verification. Each capability is first acquired by a dedicated specialist through reinforcement learning with verifiable rewards from a shared backbone, and the specialists are then consolidated into a single inference-time model through weight-space merging followed by routed policy-space distillation. We instantiate Capek 0.5 at the 2B and 35B-A3B scales and evaluate it from three complementary perspectives: comprehensive benchmark suites including Capek-StateBench, a new benchmark for state verification; a controlled study of capability retention from specialists to the unified model; and closed-loop evaluation in simulated embodied environments. Capek 0.5 improves the large majority of matched benchmark rows over its initialization, retains all four specialized capabilities in one checkpoint with quantified losses, and transfers to closed-loop embodied task execution.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 83 of its 99 references that are in the map. First posted on arXiv 7 Aug 2026.
Builds on
83 of 99 references.
Closest ancestors · 16
Same direction, newest first
- 2025NaviTrace: Evaluating Embodied Navigation of Vision-Language Models30 Oct
- 2025Point-It-Out: Benchmarking Embodied Reasoning for Vision Language Models in Multi-Stage Visual Grounding30 Sep
- 2025Towards Embodied Agentic AI: Review and Classification of LLM- and VLM-Driven Robot Autonomy and Interaction7 Aug
- 2025MindCube: Spatial Mental Modeling from Limited Views26 Jun
- 2025OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Models3 Jun
- 2025Robo2VLM: Visual Question Answering from Large-Scale In-the-Wild Robot Manipulation Datasets21 May
- 2025From Seeing to Doing: Bridging Reasoning and Decision for Robotic Manipulation13 May
- 2025From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3D29 Mar
- 2025Omnia de EgoTempo: Benchmarking Temporal Understanding of Multi-Modal LLMs in Egocentric Videos17 Mar
- 2025RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concrete28 Feb
- 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents13 Feb
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces18 Dec
4 more
- 2024RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics25 Nov
- 2024RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotics15 Jun
- 2024EmbSpatial-Bench: Benchmarking Spatial Understanding for Embodied Tasks with Large Vision-Language Models9 Jun
- 2023PaLM-E: An Embodied Multimodal Language Model6 Mar
Other references · 67
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2023Visual Instruction Tuningcited by 7,792
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
- 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
38 more
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
- 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
- 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
- 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
- 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
- 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
- 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
- 2022Editing Models with Task Arithmeticcited by 1,076
- 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
- 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 840
- 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
- 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
- 2023TIES-Merging: Resolving Interference When Merging Modelscited by 795
- 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 735
- 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
- 2017Localizing Moments in Video with Natural Languagecited by 660
- 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 644
- 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
- 2016Hollywood in Homes: Crowdsourcing Data Collection for Activity Understandingcited by 535
- 2019CLEVRER: CoLlision Events for Video REpresentation and Reasoningcited by 475
- 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 390
- 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 386
- 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
- 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 321
- 2023Perception Test: A Diagnostic Benchmark for Multimodal Video Modelscited by 310
- 2024STAR: A Benchmark for Situated Reasoning in Real-World Videoscited by 213
- 2025Cosmos-Reason1: From Physical Common Sense To Embodied Reasoningcited by 159
- 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 137
- 2023RoboVQA: Multimodal Long-Horizon Reasoning for Roboticscited by 116
- 2025Gemini Robotics 1.5: Pushing the Frontier of Generalist Robots with Advanced Embodied Reasoning, Thinking, and Motion Transfercited by 86
- 2023Hierarchical Video-Moment Retrieval and Step-Captioningcited by 83
- 2025Scaling RL to Long Videoscited by 83
- 2025Scaling Spatial Intelligence with Multimodal Foundation Modelscited by 75
- 2024VideoEspresso: A Large-Scale Chain-of-Thought Dataset for Fine-Grained Video Reasoning via Core Frame Selectioncited by 64
- 2026MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Trainingcited by 60
50 most cited of 67 All 67 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Unmasking the Illusion of Embodied Reasoning in Vision-Language-Action Models
- 2026GigaBrain-0.7: Scaling Embodied Foundation Models to Emergent Capabilities with a Three-System Architecture
- 2025Embodied4C: Measuring What Matters for Embodied Vision-Language Navigation
- 2026RoboAgent: Chaining Basic Capabilities for Embodied Task Planning
- 2026From Inference Efficiency to Embodied Efficiency: Revisiting Efficiency Metrics for Vision-Language-Action Models
- 2026HiMe: Hierarchical Embodied Memory for Long-Horizon Vision-Language-Action Control
- 2025EmbodiedBrain: Expanding Performance Boundaries of Task Planning for Embodied Intelligence
- 2026Vision-Language-Action in Robotics: A Survey of Datasets, Benchmarks, and Data Engines