Efficient Test-Time Adaptation through Human-AI Interaction
Not on the map. Why?
Abstract (arXiv)
AI agents are trained on population-scale data to encode broad capabilities spanning those of many practitioners. Yet the artifacts they produce rarely meet the personal bar professionals need to stake their reputation on. On realistic, open-ended tasks where success criteria are heterogeneous and insufficiently documented, individual expertise lives precisely in the elevation and departure from the average. In practice, iterative human-agent interaction surfaces criteria that users cannot fully specify up front, yet apply repeatedly across tasks. We argue this cross-session interaction data is a rich, underused signal for closing the gap to individual expertise. In this work, we propose test-time adaptation through human-agent interaction (TAHI), which integrates these signals into agent context and weights, and crystallizes each user's training and evaluation criteria via an evolving rubric module. We adapt agents to 30 individuals in two high-utility domains, writing and visual creation, on a total of 600 tasks. Our agents improve solo task success by 4.5-20.9% within only tens of tasks. Meanwhile, our evolving rubric module serves as a scalable annotation tool, creating evaluation rubrics that catch 16.0-22.3% more failures than those from LMs or humans alone. While agents are adapted towards individuals, we show these personalized agents also produce improvements in success of up to 8.8% that generalize across users.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 51 of its 62 references that are in the map. First posted on arXiv 3 Sep 2026.
Builds on
51 of 62 references.
References · 51
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2017Deep reinforcement learning from human preferencescited by 3,846
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
- 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,366
- 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
- 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
- 2025A-MEM: Agentic Memory for LLM Agentscited by 1,000
- 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
38 more
- 2026Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Modelscited by 386
- 2024Agent Workflow Memorycited by 342
- 2024TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Taskscited by 306
- 2024Training Software Engineering Agents and Verifiers with SWE-Gymcited by 300
- 2026SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learningcited by 252
- 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RLcited by 212
- 2025SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skillscited by 174
- 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 155
- 2025OpenCUA: Open Foundations for Computer-Use Agentscited by 151
- 2024Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferencescited by 142
- 2025GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Taskscited by 142
- 2025Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)cited by 137
- 2024MaxMin-RLHF: Alignment with Diverse Human Preferencescited by 121
- 2024Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinementcited by 111
- 2024Understanding the performance gap between online and offline alignment algorithmscited by 110
- 2025Inducing Programmatic Skills for Agentic Taskscited by 103
- 2026Learning to Discover at Test Timecited by 94
- 2026OpenClaw-RL: Train Any Agent Simply by Talkingcited by 83
- 2024RLHF from Heterogeneous Feedback via Personalization and Preference Aggregationcited by 71
- 2022On the Limitations of Reference-Free Evaluations of Generated Textcited by 52
- 2026Expanding the Capabilities of Reinforcement Learning via Text Feedbackcited by 49
- 2024Synatra: Turning Indirect Knowledge into Direct Demonstrations for Digital Agents at Scalecited by 43
- 2025Toward an Evaluation Science for Generative AI Systemscited by 42
- 2024BAGEL: Bootstrapping Agents by Guiding Exploration with Languagecited by 34
- 2025Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interactioncited by 34
- 2024Aligning Language Models with Demonstrated Feedbackcited by 32
- 2026APEX-Agentscited by 29
- 2019Continual adaptation for efficient machine communicationcited by 28
- 2026Gym-Anything: Turn any Software into an Agent Environmentcited by 24
- 2026Aligning Language Models from User Interactionscited by 21
- 2026How LLMs Distort Our Written Languagecited by 20
- 2021Continual Learning for Grounded Instruction Generation by Observing Human Following Behaviorcited by 19
- 2025Efficient Agent Training for Computer Usecited by 17
- 2025Continual Learning via Sparse Memory Finetuningcited by 17
- 2025How Do AI Agents Do Human Work? Comparing AI and Human Workflows Across Diverse Occupationscited by 17
- 2025Agent Data Protocol: Unifying Datasets for Diverse, Effective Fine-tuning of LLM Agentscited by 16
- 2026Learning, Fast and Slow: Towards LLMs That Adapt Continuallycited by 6
- 2026CollabSkill: Evaluating Human-Agent Collaboration On Real-World Taskscited by 1
50 most cited of 51 All 51 (JSON)
Built on by
Not cited yet.