mlsanityData to 14 Oct 2026

VIEWCRAFT: Evolving What Multimodal Agents See and How They Read It

arXiv 2610.15915 ↗First posted 11 Oct 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Reusable skills improve multimodal agents without updating model weights, but their effectiveness depends on both the evidence presented and the procedure used to read it. We introduce ViewCraft, which jointly evolves an executable evidence interface and its reading procedure. The interface selects, transforms, and organizes observations; the reader interprets the resulting views. Failure traces guide revisions, replay and paired evaluation retain useful changes, and interface projection recovers visual edits from eligible rejected joint proposals. Across eight benchmarks, ViewCraft improves over direct execution by an average of 6.1 points in task score, with gains on every benchmark. The advantage persists at a matched input-token budget. On all five multi-source tasks, the learned interface uses 8%-15% fewer input tokens than Initial skill. The learned skills also transfer across executors and outperform direct execution in every evaluated transfer, supporting evidence construction and reading as a coupled, reusable skill.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 48 of its 55 references that are in the map. First posted on arXiv 11 Oct 2026.

Builds on

48 of 55 references.

References · 48

  1. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  2. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  3. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  4. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 876
  5. 2023Large Language Models as Optimizerscited by 864
  6. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 551
  7. 2025GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learningcited by 542
  8. 2024AFlow: Automating Agentic Workflow Generationcited by 432
  9. 2024Automated Design of Agentic Systemscited by 407
  10. 2019FUNSD: A Dataset for Form Understanding in Noisy Scanned Documentscited by 349
  11. 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 332
  12. 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Modelscited by 268
36 more
  1. 2024VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videoscited by 251
  2. 2023SlideVQA: A Dataset for Document Visual Question Answering on Multiple Imagescited by 179
  3. 2024AgentSquare: Automatic LLM Agent Search in Modular Design Spacecited by 155
  4. 2026SkillOpt: Executive Strategy for Self-Evolving Agent Skillscited by 148
  5. 2025Video-Holmes: Can MLLM Think Like Holmes for Complex Video Reasoning?cited by 118
  6. 2024Frame-Voyager: Learning to Query Frames for Video Large Language Modelscited by 83
  7. 2025STI-Bench: Are MLLMs Ready for Precise Spatial-Temporal World Understanding?cited by 79
  8. 2026XSkill: Continual Learning from Experience and Skills in Multimodal Agentscited by 75
  9. 2024MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Modelscited by 67
  10. 2025UniCTokens: Boosting Personalized Understanding and Generation via Unified Concept Tokenscited by 56
  11. 2024MC-LLaVA: Multi-Concept Personalized Vision-Language Modelcited by 54
  12. 2025Perceive Anything: Recognize, Explain, Caption, and Segment Anything in Images and Videoscited by 51
  13. 2024V2Xum-LLM: Cross-Modal Video Summarization with Temporal Prompt Instruction Tuningcited by 46
  14. 2024LLM as Dataset Analyst: Subpopulation Structure Discovery with Large Language Modelcited by 44
  15. 2024MMCOMPOSITION: Revisiting the Compositionality of Pre-trained Vision-Language Modelscited by 31
  16. 2024FINECAPTION: Compositional Image Captioning Focusing on Wherever You Want at Any Granularitycited by 27
  17. 2026M2A: Multimodal Memory Agent with Dual-Layer Hybrid Memory for Long-Term Personalized Interactionscited by 25
  18. 2025ReasonMap: Towards Fine-Grained Visual Reasoning from Transit Mapscited by 23
  19. 2024FINEMATCH: Aspect-based Fine-grained Image and Text Mismatch Detection and Correctioncited by 17
  20. 2025MIRA: Multimodal Iterative Reasoning Agent for Image Editingcited by 17
  21. 2026MMSkills: Towards Multimodal Skills for General Visual Agentscited by 16
  22. 2026GENIUS: Generative Fluid Intelligence Evaluation Suitecited by 12
  23. 2026WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolutioncited by 12
  24. 2026MementoGUI: Learning Agentic Multimodal Memory Control for Long-Horizon GUI Agentscited by 11
  25. 2025Caption Anything in Video: Fine-grained Object-centric Captioning via Spatiotemporal Multimodal Promptingcited by 10
  26. 2026PEARL: Personalized Streaming Video Understanding Modelcited by 9
  27. 2026Aurora: Unified Video Editing with a Tool-Using Agentcited by 9
  28. 2025Multimodal Prompt Optimization: Why Not Leverage Multiple Modalities for MLLMscited by 7
  29. 2026Do Less, Achieve More: Do We Need Every-Step Optimization for RL Fine-tuning of Diffusion Models?cited by 6
  30. 2025Small-Large Collaboration: Training-efficient Concept Personalization for Large VLM using a Meta Personalized Small VLMcited by 4
  31. 2025Jarvis: Towards Personalized AI Assistant via Personal KV-Cache Retrievalcited by 4
  32. 2026AD-MIR: Bridging the Gap from Perception to Persuasion in Advertising Video Understanding via Structured Reasoningcited by 2
  33. 2026Agent Skills Should Go Beyond Text: The Case for Visual Skillscited by 2
  34. 2026Dynamo: Dynamic Skill-Tool Evolution for Vision-Language Agentscited by 2
  35. 2026Pixel-Space Diffusion Transformerscited by 1
  36. 2026EpiCon: Collective Agent Learning through Co-Evolving Multimodal Memorycited by 1

Built on by

Not cited yet.