Omni-IO Skills: Harnessing Your Agent Omni-Native
On the map: LLM agents and tool use → Software engineering coding agents also fits: Agent skill learning and evolution
Abstract (arXiv)
General-purpose agents can plan, reason, and act over long horizons, yet their production capabilities remain fragmented across text, images, audio, video, documents, 3D assets, and code. Extending a foundation model to additional modalities ties capability growth to costly model updates, while assembling specialist models and tools leaves unresolved how procedures, dependencies, intermediate assets, and cross-turn revisions should be coordinated. We present Omni-IO Skills, a plug-and-play Agent Harness that makes existing agents omni-native through hierarchical Skills, a standardized multimodal execution interface, dependency-aware orchestration, and a persistent Asset Registry. Multi-asset workflows are represented as Declare Execution Graphs, which schedule independent operations concurrently and register successful outputs for downstream and cross-turn reuse across replaceable execution backends. Its 27 Skills cover 38 representative tasks spanning seven artifact modalities and four capability families: understanding, generation, reasoning, and retrieval. On UniM-90, the harness raises the input-support rates of GPT-5.6 Sol and Claude Sonnet 5 from 40.00% and 38.89% to 100%, while increasing relative Semantic--Quality Coupled Score from 26.99 to 74.94 and from 27.82 to 77.78, respectively; Strict Structure Score reaches 100.00 and 99.78. These results establish harness-level capability composition as a practical route to broad, evolvable Omni systems without changing the host agent's reasoning core.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 30 of its 35 references that are in the map. First posted on arXiv 25 Sep 2026.
Builds on
30 of 35 references.
References · 30
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
- 2022Video Diffusion Modelscited by 2,280
- 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
- 2025Qwen2.5-Omni Technical Reportcited by 704
- 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
- 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
- 2024Moshi: a speech-text foundation model for real-time dialoguecited by 536
- 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 511
- 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
18 more
- 2025Qwen3-Omni Technical Reportcited by 451
- 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
- 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 301
- 2026SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Taskscited by 289
- 2024AnyGPT: Unified Multimodal LLM with Discrete Sequence Modelingcited by 213
- 2024JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generationcited by 162
- 2025Ming-Omni: A Unified Multimodal Model for Perception and Generationcited by 62
- 2026CUA-Skill: Develop Skills for Computer Using Agentcited by 32
- 2026OmniGAIA: Towards Native Omni-Modal AI Agentscited by 21
- 2024PanoSent: A Panoptic Sextuple Extraction Benchmark for Multimodal Conversational Aspect-based Sentiment Analysiscited by 13
- 2025Dr.V: A Hierarchical Perception-Temporal-Cognition Framework to Diagnose Video Hallucination by Fine-grained Spatial-Temporal Groundingcited by 13
- 2026MMSkills: Towards Multimodal Skills for General Visual Agentscited by 13
- 2025Agent-Omni: Test-Time Multimodal Reasoning via Model Coordination for Understanding Anythingcited by 9
- 2026UniM: A Unified Any-to-Any Interleaved Multimodal Benchmarkcited by 9
- 2026Unveiling the Cognitive Compass: Theory-of-Mind-Guided Multimodal Emotion Reasoningcited by 8
- 2025VEGAS: Towards Visually Explainable and Grounded Artificial Social Intelligencecited by 4
- 2026Towards Comprehensive Stage-wise Benchmarking of Large Language Models in Fact-Checkingcited by 3
- 2026SafeGuard: A Multi-Agent Perception-Reasoning Framework for Social-Risk AI-Generated Video Detectioncited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Qwen3.8-Omni: Towards Native Omni-Modal Agents
- 2026Agent Skills: A Data-Driven Analysis of Claude Skills for Extending Large Language Model Functionality
- 2026Subagents vs Agent Skills: Executing Reusable Knowledge for Long-Horizon Agentic Tasks
- 2026OmniaBench: Benchmarking General AI Agents Across Diverse Scenarios
- 2026A Comprehensive Survey on Agent Skills: Taxonomy, Techniques, and Applications
- 2026SkillSmith: Compiling Agent Skills into Boundary-Guided Runtime Interfaces
- 2026Evaluating Agent Skills for Version-Specific Plugin Migration: A Retrospective Study
- 2026SkillAlign: Aligning Skill Interfaces for LLM-based Agents