mlsanityData to 14 Oct 2026

Echo: Learning from Experience Data via User-Driven Refinement

arXiv 2605.21984 ↗First posted 21 May 2026cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Static "human data" faces inherent limitations: it is expensive to scale and bounded by the knowledge of its creators. Continuous learning from "experience data" - interactions between agents and their environments - promises to transcend these barriers. Today, the widespread deployment of AI agents grants us low-cost access to massive streams of such real-world experience. However, raw interaction logs are inherently noisy, filled with trial-and-error and low information density, rendering them inefficient for direct model training. We introduce Echo, a generalized framework designed to operationalize the transition from raw experience to learnable knowledge, effectively "echoing" environmental feedback back into the training loop for model optimization. In today's agent ecosystem, user refinement serves as a primary source of such feedback: driven by responsibility for the outcome, users rigorously transform flawed agent proposals into verified solutions. These user-driven refinement sequences inherently distill agents' crude attempts into high-quality training signals. Echo systematically harvests these signals to continuously align the agent with real-world needs. Large-scale validation in a production code completion environment confirms that Echo effectively harnesses this pipeline, breaking the static performance ceiling by increasing the acceptance rate from 25.7% to 35.7%.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 45 references that are in the map. First posted on arXiv 21 May 2026.

Builds on

31 of 45 references.

References · 31

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2020Scaling Laws for Neural Language Modelscited by 6,586
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  6. 2017Deep reinforcement learning from human preferencescited by 3,912
  7. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  8. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  9. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
  10. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
  11. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
  12. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
19 more
  1. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 759
  2. 2021Deduplicating Training Data Makes Language Models Bettercited by 714
  3. 2023Textbooks Are All You Needcited by 569
  4. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 548
  5. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 498
  6. 2020CodeBLEU: a Method for Automatic Evaluation of Code Synthesiscited by 479
  7. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 467
  8. 2019Challenges of Real-World Reinforcement Learningcited by 291
  9. 2022Efficient Training of Language Models to Fill in the Middlecited by 255
  10. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 205
  11. 2024RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMscited by 90
  12. 2025Agent Learning via Early Experiencecited by 85
  13. 2023CoEdIT: Text Editing by Task-Specific Instruction Tuningcited by 68
  14. 2025Scaling Agent Learning via Experience Synthesiscited by 44
  15. 2025Measuring Agents in Productioncited by 37
  16. 2026Online Experiential Learning for Language Modelscited by 28
  17. 2023AI-assisted Code Authoring at Scale: Fine-tuning, deploying, and mixed methods evaluationcited by 21
  18. 2025Experience with GitHub Copilot for Developer Productivity at Zoominfocited by 8
  19. 2025Reinforcement Learning from User Feedbackcited by 8

Built on by

Not cited yet.