# Coding Agents with Multimodal Browsing are Generalist Problem Solvers

arXiv: https://arxiv.org/abs/2506.03011 · HTML: https://mlsanity.com/paper/2506.03011/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 15 of its 34 references that are in the map.
- First posted on arXiv 3 Jun 2025.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 15 of 15)

- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments](https://mlsanity.com/paper/2404.07972/) (2024)
- [GAIA: a benchmark for General AI Assistants](https://mlsanity.com/paper/2311.12983/) (2023)
- [OpenHands: An Open Platform for AI Software Developers as Generalist Agents](https://mlsanity.com/paper/2407.16741/) (2024)
- [Agentless: Demystifying LLM-based Software Engineering Agents](https://mlsanity.com/paper/2407.01489/) (2024)
- [Agent Workflow Memory](https://mlsanity.com/paper/2409.07429/) (2024)
- [Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks](https://mlsanity.com/paper/2411.04468/) (2024)
- [SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains?](https://mlsanity.com/paper/2410.03859/) (2024)
- [The BrowserGym Ecosystem for Web Agent Research](https://mlsanity.com/paper/2412.05467/) (2024)
- [OWL: Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automation](https://mlsanity.com/paper/2505.23885/) (2025)
- [AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents](https://mlsanity.com/paper/2410.13825/) (2024)
- [Symbiotic Cooperation for Web Agents: Harnessing Complementary Strengths of Large and Small LLMs](https://mlsanity.com/paper/2502.07942/) (2025)
- [TapeAgents: a Holistic Framework for Agent Development and Optimization](https://mlsanity.com/paper/2412.08445/) (2024)

## Built on by (most cited first, 25 of 25)

- [The OpenHands Software Agent SDK: A Composable and Extensible Foundation for Production Agents](https://mlsanity.com/paper/2511.03690/) (2025)
- [How Do AI Agents Do Human Work? Comparing AI and Human Workflows Across Diverse Occupations](https://mlsanity.com/paper/2510.22780/) (2025)
- [InnovatorBench: Evaluating Agents' Ability to Conduct Innovative LLM Research](https://mlsanity.com/paper/2510.27598/) (2025)
- [Temporary Authority, Permanent Effects: Commit-Time Authorization for LLM Agents](https://mlsanity.com/paper/2607.10487/) (2026)
- [Open Data Synthesis For Deep Research](https://mlsanity.com/paper/2509.00375/) (2025)
- [WebGen-Agent: Enhancing Interactive Website Generation with Multi-Level Feedback and Step-Level Reinforcement Learning](https://mlsanity.com/paper/2509.22644/) (2025)
- [A Comprehensive Survey on Benchmarks and Solutions in Software Engineering of LLM-Empowered Agentic System](https://mlsanity.com/paper/2510.09721/) (2025)
- [Can Language Models Discover Scaling Laws?](https://mlsanity.com/paper/2507.21184/) (2025)
- [SERA: Soft-Verified Efficient Repository Agents](https://mlsanity.com/paper/2601.20789/) (2026)
- [CodeScout: An Effective Recipe for Reinforcement Learning of Code Search Agents](https://mlsanity.com/paper/2603.17829/) (2026)
- [SkillMaster: Toward Autonomous Skill Mastery in LLM Agents](https://mlsanity.com/paper/2605.08693/) (2026)
- [Completion neq Collaboration: Scaling Collaborative Effort with Agents](https://mlsanity.com/paper/2510.25744/) (2025)
- [Agentic Software Issue Resolution with Large Language Models: A Survey](https://mlsanity.com/paper/2512.22256/) (2025)
- [MetaAgent: Toward Self-Evolving Agent via Tool Meta-Learning](https://mlsanity.com/paper/2508.00271/) (2025)
- [How can we assess human-agent interactions? Case studies in software agent design](https://mlsanity.com/paper/2510.09801/) (2025)
- [TheMCPCompany: Creating General-purpose Agents with Task-specific Tools](https://mlsanity.com/paper/2510.19286/) (2025)
- [Humans are Missing from AI Coding Agent Research](https://mlsanity.com/paper/2608.12355/) (2026)
- [Library Hallucinations in LLM-Generated Code: A Risk Analysis Grounded in Developer Queries](https://mlsanity.com/paper/2509.22202/) (2025)
- [From Refusal to Recovery: A Control-Theoretic Approach to Generative AI Guardrails](https://mlsanity.com/paper/2510.13727/) (2025)
- [Towards Reliable ML Feature Engineering via Planning in Constrained-Topology of LLM Agents](https://mlsanity.com/paper/2601.10820/) (2026)
- [Can Current Agents Close the Discovery-to-Application Gap? A Case Study in Minecraft](https://mlsanity.com/paper/2604.24697/) (2026)
- [CVEvolve: Autonomous Algorithm Discovery for Unstructured Scientific Data Processing](https://mlsanity.com/paper/2605.11359/) (2026)
- [Towards Direct Latent-Space Synthesis for Parallel Branches in LLM-Agent Workflows](https://mlsanity.com/paper/2606.14672/) (2026)
- [Data Intelligence Agents: Interpreting, Modeling, and Querying Enterprise Data via Autonomous Coding Agents](https://mlsanity.com/paper/2606.19319/) (2026)
- [The Last Mile of Coding Agents for Long-Document Tasks: An Empirical Study on Clinical Trial Protocol Digitization](https://mlsanity.com/paper/2610.18507/) (2026)

## Abstract (arXiv)

Modern human labor is characterized by specialization; we train for years and develop particular tools that allow us to perform well across a variety of tasks. In addition, AI agents have been specialized for domains such as software engineering, web navigation, and workflow automation. However, this results in agents that are good for one thing but fail to generalize beyond their intended scope. One reason for this is that agent developers provide a highly specialized set of tools or make architectural decisions optimized for a specific use case or benchmark. In this work, we ask the question: what is the minimal set of general tools that can be used to achieve high performance across a diverse set of tasks? Our answer is OpenHands-Versa, a generalist agent built with a modest number of general tools: code editing and execution, web search, as well as multimodal web browsing and file access. Importantly, OpenHands-Versa demonstrates superior or competitive performance over leading specialized agents across three diverse and challenging benchmarks: SWE-Bench Multimodal, GAIA, and The Agent Company, outperforming the best-performing previously published results with absolute improvements in success rate of 9.1, 1.3, and 9.1 points respectively. Further, we show how existing state-of-the-art multi-agent systems fail to generalize beyond their target domains. These results demonstrate the feasibility of developing a generalist agent to solve diverse tasks and establish OpenHands-Versa as a strong baseline for future research.