# Orca: Browsing at Scale Through User-Driven and AI-Facilitated Orchestration Across Malleable Webpages

arXiv: https://arxiv.org/abs/2505.22831 · HTML: https://mlsanity.com/paper/2505.22831/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 115 references that are in the map.
- First posted on arXiv 28 May 2025.

## Builds on (most cited first, 19 of 19)

- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [Mind2Web: Towards a Generalist Agent for the Web](https://mlsanity.com/paper/2306.06070/) (2023)
- [WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models](https://mlsanity.com/paper/2401.13919/) (2024)
- [Bridging the Gulf of Envisioning: Cognitive Design Challenges in LLM Interfaces](https://mlsanity.com/paper/2309.14459/) (2023)
- [Graphologue: Exploring Large Language Model Responses with Interactive Diagrams](https://mlsanity.com/paper/2305.11473/) (2023)
- [Sensecape: Enabling Multilevel Exploration and Sensemaking with Large Language Models](https://mlsanity.com/paper/2305.11483/) (2023)
- [CowPilot: A Framework for Autonomous and Human-Agent Collaborative Web Navigation](https://mlsanity.com/paper/2501.16609/) (2025)
- [Improving Steering and Verification in AI-Assisted Data Analysis with Interactive Task Decomposition](https://mlsanity.com/paper/2407.02651/) (2024)
- [Generative and Malleable User Interfaces with Generative and Evolving Task-Driven Data Model](https://mlsanity.com/paper/2503.04084/) (2025)
- [Search Engines in an AI Era: The False Promise of Factual and Verifiable Source-Cited Responses](https://mlsanity.com/paper/2410.22349/) (2024)
- [Textoshop: Interactions Inspired by Drawing Software to Facilitate Text Editing](https://mlsanity.com/paper/2409.17088/) (2024)
- [Crystalline: Lowering the Cost for Developers to Collect and Organize Information for Decision Making](https://mlsanity.com/paper/2202.02175/) (2022)
- [Malleable Overview-Detail Interfaces](https://mlsanity.com/paper/2503.07782/) (2025)
- [Fuse: In-Situ Sensemaking Support in the Browser](https://mlsanity.com/paper/2208.14861/) (2022)
- [CrossTalk: Intelligent Substrates for Language-Oriented Interaction in Video-Based Communication and Collaboration](https://mlsanity.com/paper/2308.03311/) (2023)
- [Wigglite: Low-cost Information Collection and Triage](https://mlsanity.com/paper/2208.00496/) (2022)
- [DiLogics: Creating Web Automation Programs With Diverse Logics](https://mlsanity.com/paper/2308.05828/) (2023)
- [Feedforward in Generative AI: Opportunities for a Design Space](https://mlsanity.com/paper/2502.14229/) (2025)
- [DOBBS: Towards a Comprehensive Dataset to Study the Browsing Behavior of Online Users](https://mlsanity.com/paper/1307.1542/) (2013)

## Built on by (most cited first, 5 of 5)

- [Interaction-Driven Browsing: A Human-in-the-Loop Conceptual Framework Informed by Human Web Browsing for Browser-Using Agents](https://mlsanity.com/paper/2509.12049/) (2025)
- [On the Regulatory Potential of User Interfaces for AI Agent Governance](https://mlsanity.com/paper/2512.00742/) (2025)
- ["When to Hand Off, When to Work Together": Understanding Concurrent Human-Agent Interaction in Shared Co-Creative Workspaces](https://mlsanity.com/paper/2603.02050/) (2026)
- [Sidekick: Designing Communication for Effective Multitasking with Computer Use Agents](https://mlsanity.com/paper/2607.17527/) (2026)
- [Task-State Governance: A Virtual Machine for Verifiable and Recoverable GUI Agent Execution](https://mlsanity.com/paper/2610.20100/) (2026)

## Abstract (arXiv)

Web-based activities span multiple webpages. However, conventional browsers with stacks of tabs cannot support operating and synthesizing large volumes of information across pages. While recent AI systems enable fully automated web browsing and information synthesis, they often diminish user agency and hinder contextual understanding. We explore how AI could instead augment user interactions with content across webpages and mitigate cognitive and manual efforts. Through literature on information tasks and web browsing challenges, and an iterative design process, we present novel interactions with our prototype web browser, Orca. Leveraging AI, Orca supports user-driven exploration, operation, organization, and synthesis of web content at scale. To enable browsing at scale, webpages are treated as malleable materials that humans and AI can collaboratively manipulate and compose into a malleable, dynamic, and browser-level workspace. Our evaluation revealed an increased "appetite" for information foraging, enhanced control, and more flexible sensemaking across a broader web information landscape.