# ALLOY: Generating Reusable Agent Workflows from User Demonstration

arXiv: https://arxiv.org/abs/2510.10049 · HTML: https://mlsanity.com/paper/2510.10049/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 16 of its 70 references that are in the map.
- First posted on arXiv 11 Oct 2025.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 16 of 16)

- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Models](https://mlsanity.com/paper/2503.23350/) (2025)
- [Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agents](https://mlsanity.com/paper/2402.09205/) (2024)
- [Bridging the Gulf of Envisioning: Cognitive Design Challenges in LLM Interfaces](https://mlsanity.com/paper/2309.14459/) (2023)
- [Learning to Navigate the Web](https://mlsanity.com/paper/1812.09195/) (2018)
- [Agentic Web: Weaving the Next Web with AI Agents](https://mlsanity.com/paper/2507.21206/) (2025)
- [Aligning Language Models with Demonstrated Feedback](https://mlsanity.com/paper/2406.00888/) (2024)
- [Flow: Modularized Agentic Workflow Automation](https://mlsanity.com/paper/2501.07834/) (2025)
- [DOM-LM: Learning Generalizable Representations for HTML Documents](https://mlsanity.com/paper/2201.10608/) (2022)
- [AdaptAgent: Adapting Multimodal Web Agents with Few-Shot Learning from Human Demonstrations](https://mlsanity.com/paper/2411.13451/) (2024)
- [Dark Patterns Meet GUI Agents: LLM Agent Susceptibility to Manipulative Interfaces and the Role of Human Oversight](https://mlsanity.com/paper/2509.10723/) (2025)
- [Characterizing Unintended Consequences in Human-GUI Agent Collaboration for Web Browsing](https://mlsanity.com/paper/2505.09875/) (2025)
- [ChainBuddy: An AI Agent System for Generating LLM Pipelines](https://mlsanity.com/paper/2409.13588/) (2024)
- [Show or Tell? Demonstration is More Robust to Changes in Shared Perception than Explanation](https://mlsanity.com/paper/2012.09035/) (2020)

## Built on by (most cited first, 3 of 3)

- [PV-SQL: Synergizing Database Probing and Rule-based Verification for Text-to-SQL Agents](https://mlsanity.com/paper/2604.17653/) (2026)
- [How Should Agents Read Demonstrations? Hierarchical Structure Beats Flat Action Logs](https://mlsanity.com/paper/2606.20978/) (2026)
- [Motif: Discovering and Automating Personal Web Workflows](https://mlsanity.com/paper/2607.10531/) (2026)

## Abstract (arXiv)

Large language models (LLMs) enable end-users to delegate complex tasks to autonomous agents through natural language. However, prompt-based interaction faces critical limitations: Users often struggle to specify procedural requirements for tasks, especially those that don't have a factually correct solution but instead rely on personal preferences, such as posting social media content or planning a trip. Additionally, a ''successful'' prompt for one task may not be reusable or generalizable across similar tasks. We present ALLOY, a system inspired by classical HCI theories on Programming by Demonstration (PBD), but extended to enhance adaptability in creating LLM-based web agents. ALLOY enables users to express procedural preferences through natural demonstrations rather than prompts, while making these procedures transparent and editable through visualized workflows that can be generalized across task variations. In a study with 12 participants, ALLOY's demonstration--based approach outperformed prompt-based agents and manual workflows in capturing user intent and procedural preferences in complex web tasks. Insights from the study also show how demonstration--based interaction complements the traditional prompt-based approach.