# Role-RL: Online Long-Context Processing with Role Reinforcement Learning for Distinct LLMs in Their Optimal Roles

arXiv: https://arxiv.org/abs/2409.18014 · HTML: https://mlsanity.com/paper/2409.18014/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 24 references that are in the map.
- First posted on arXiv 26 Sep 2024.

## Builds on (most cited first, 18 of 18)

- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate](https://mlsanity.com/paper/2305.19118/) (2023)
- [Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents](https://mlsanity.com/paper/2306.03314/) (2023)
- [LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens](https://mlsanity.com/paper/2402.13753/) (2024)
- [A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaboration](https://mlsanity.com/paper/2310.02170/) (2023)
- [Chain of Agents: Large Language Models Collaborating on Long-Context Tasks](https://mlsanity.com/paper/2406.02818/) (2024)
- [LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning](https://mlsanity.com/paper/2401.01325/) (2024)
- [PRD: Peer Rank and Discussion Improve Large Language Model based Evaluations](https://mlsanity.com/paper/2307.02762/) (2023)
- [Embodied LLM Agents Learn to Cooperate in Organized Teams](https://mlsanity.com/paper/2403.12482/) (2024)
- [Infinite-LLM: Efficient LLM Service for Long Context with DistAttention and Distributed KVCache](https://mlsanity.com/paper/2401.02669/) (2024)
- [PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documents](https://mlsanity.com/paper/2305.14564/) (2023)
- [Do LLMs Have Distinct and Consistent Personality? TRAIT: Personality Testset designed for LLMs with Psychometrics](https://mlsanity.com/paper/2406.14703/) (2024)
- [Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length](https://mlsanity.com/paper/2404.08801/) (2024)
- [LongAgent: Scaling Language Models to 128k Context through Multi-Agent Collaboration](https://mlsanity.com/paper/2402.11550/) (2024)
- [RL-GPT: Integrating Reinforcement Learning and Code-as-policy](https://mlsanity.com/paper/2402.19299/) (2024)
- [MLLM-CompBench: A Comparative Reasoning Benchmark for Multimodal LLMs](https://mlsanity.com/paper/2407.16837/) (2024)
- [AgentCoord: Visually Exploring Coordination Strategy for LLM-based Multi-Agent Collaboration](https://mlsanity.com/paper/2404.11943/) (2024)
- [Exploring the Capabilities of LLMs for Code Change Related Tasks](https://mlsanity.com/paper/2407.02824/) (2024)

## Abstract (arXiv)

Large language models (LLMs) with long-context processing are still challenging because of their implementation complexity, training efficiency and data sparsity. To address this issue, a new paradigm named Online Long-context Processing (OLP) is proposed when we process a document of unlimited length, which typically occurs in the information reception and organization of diverse streaming media such as automated news reporting, live e-commerce, and viral short videos. Moreover, a dilemma was often encountered when we tried to select the most suitable LLM from a large number of LLMs amidst explosive growth aiming for outstanding performance, affordable prices, and short response delays. In view of this, we also develop Role Reinforcement Learning (Role-RL) to automatically deploy different LLMs in their respective roles within the OLP pipeline according to their actual performance. Extensive experiments are conducted on our OLP-MINI dataset and it is found that OLP with Role-RL framework achieves OLP benchmark with an average recall rate of 93.2% and the LLM cost saved by 79.4%. The code and dataset are publicly available at: https://anonymous.4open.science/r/Role-RL.