# RIZZ: Routing Interactions to Near Zero-Interference Zones for Continual Adaptation of Black-Box Agents

arXiv: https://arxiv.org/abs/2606.20638 · HTML: https://mlsanity.com/paper/2606.20638/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Agent memory systems, one of 16 directions in LLM agents and tool use.
- Builds on 31 of its 41 references that are in the map.
- First posted on arXiv 2 Jun 2026.

## Placed in

- LLM agents and tool use / [Agent memory systems](https://mlsanity.com/agents/agent-memory-systems/)

## Builds on (most cited first, 30 of 31)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!](https://mlsanity.com/paper/2310.03693/) (2023)
- [Learning to Prompt for Continual Learning](https://mlsanity.com/paper/2112.08654/) (2021)
- [DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines](https://mlsanity.com/paper/2310.03714/) (2023)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning](https://mlsanity.com/paper/2507.19457/) (2025)
- [Understanding the planning of LLM agents: A survey](https://mlsanity.com/paper/2402.02716/) (2024)
- [Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Ranking](https://mlsanity.com/paper/2601.04720/) (2026)
- [Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory](https://mlsanity.com/paper/2504.07952/) (2025)
- [TRACE: A Comprehensive Benchmark for Continual Learning in Large Language Models](https://mlsanity.com/paper/2310.06762/) (2023)
- [StreamBench: Towards Benchmarking Continuous Improvement of Language Agents](https://mlsanity.com/paper/2406.08747/) (2024)
- [Feedback Descent: Open-Ended Text Optimization via Pairwise Comparison](https://mlsanity.com/paper/2511.07919/) (2025)
- [Maintaining Plasticity in Deep Continual Learning](https://mlsanity.com/paper/2306.13812/) (2023)
- [Rethinking Data Selection for Supervised Fine-Tuning](https://mlsanity.com/paper/2402.06094/) (2024)
- [ARES: Alternating Reinforcement Learning and Supervised Fine-Tuning for Enhanced Multi-Modal Chain-of-Thought Reasoning Through Diverse AI Feedback](https://mlsanity.com/paper/2407.00087/) (2024)
- [Continual task learning in natural and artificial agents](https://mlsanity.com/paper/2210.04520/) (2022)
- [Sweeping Heterogeneity with Smart MoPs: Mixture of Prompts for LLM Task Adaptation](https://mlsanity.com/paper/2310.02842/) (2023)
- [QCalEval: Benchmarking Vision-Language Models for Quantum Calibration Plot Understanding](https://mlsanity.com/paper/2604.25884/) (2026)
- [Continual Learning Using Only Large Language Model Prompting](https://mlsanity.com/paper/2412.15479/) (2024)
- [MetaSym: A Symplectic Meta-learning Framework for Physical Intelligence](https://mlsanity.com/paper/2502.16667/) (2025)
- [Meta-learning characteristics and dynamics of quantum systems](https://mlsanity.com/paper/2503.10492/) (2025)
- [On the Plasticity and Stability for Post-Training Large Language Models](https://mlsanity.com/paper/2602.06453/) (2026)
- [ELLA: Efficient Lifelong Learning for Adapters in Large Language Models](https://mlsanity.com/paper/2601.02232/) (2026)

## Built on by (most cited first, 1 of 1)

- [Decoupling Memory from Context: Structured Memory for Token-Efficient Test-Time Continual Learning](https://mlsanity.com/paper/2610.02687/) (2026)

## Abstract (arXiv)

Large language models are increasingly deployed as long-lived agents that must adapt across users, tasks, domains, modalities, and feedback regimes without access to model weights. Existing black-box adaptation methods typically optimize a single prompt, maintain an undifferentiated memory, or rely on repeated rollout-heavy search. However, these designs struggle when streams of input are nonstationary, feedback is sparse, and failures from one task family can contaminate behavior on another. We introduce RIZZ (Routing Interactions to Near Zero-interference Zones), a continual adaptation framework for compound language-model systems that learns entirely through verifier-gated memory, routing, and prompt compilation. RIZZ organizes input streams into dynamically spawned memory branches. At inference time, either while online or offline, a context-aware router selects or creates a branch that retrieves branch-local, global, graph-structured, and working-memory context, which is compiled into a bounded prompt together with retrieved task evidence. After the model acts, task verifiers score the output, and only verified interactions can update memory, promote reusable rules, demote harmful rules, or create anti-patterns. This yields a black-box agent that improves through persistent natural-language feedback while explicitly controlling interference. RIZZ targets the regime where adaptation must occur online under context budgets. Finally, we demonstrate the effectiveness of our framework against state-of-the-art baselines on competitive benchmarks.