# A Schema Bounded Language Model for Refining Robot Policies Without Destabilizing Local Learning

arXiv: https://arxiv.org/abs/2609.05133 · HTML: https://mlsanity.com/paper/2609.05133/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 24 references that are in the map.
- First posted on arXiv 4 Sep 2026.

## Builds on (most cited first, 16 of 16)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Phi-4 Technical Report](https://mlsanity.com/paper/2412.08905/) (2024)
- [Building Cooperative Embodied Agents Modularly with Large Language Models](https://mlsanity.com/paper/2307.02485/) (2023)
- [RoCo: Dialectic Multi-Robot Collaboration with Large Language Models](https://mlsanity.com/paper/2307.04738/) (2023)
- [SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models](https://mlsanity.com/paper/2309.10062/) (2023)
- [Scalable Multi-Robot Collaboration with Large Language Models: Centralized or Decentralized Systems?](https://mlsanity.com/paper/2309.15943/) (2023)
- [COHERENT: Collaboration of Heterogeneous Multi-Robot System with Large Language Models](https://mlsanity.com/paper/2409.15146/) (2024)
- [EMOS: Embodiment-aware Heterogeneous Multi-robot Operating System with LLM Agents](https://mlsanity.com/paper/2410.22662/) (2024)
- [From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems](https://mlsanity.com/paper/2405.19883/) (2024)
- [LAMARL: LLM-Aided Multi-Agent Reinforcement Learning for Cooperative Policy Generation](https://mlsanity.com/paper/2506.01538/) (2025)
- [MHRC: Closed-loop Decentralized Multi-Heterogeneous Robot Collaboration with Large Language Models](https://mlsanity.com/paper/2409.16030/) (2024)
- [Language-Conditioned Offline RL for Multi-Robot Navigation](https://mlsanity.com/paper/2407.20164/) (2024)
- [YOLO-MARL: You Only LLM Once for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2410.03997/) (2024)
- [GenSwarm: Scalable Multi-Robot Code-Policy Generation and Deployment via Language Models](https://mlsanity.com/paper/2503.23875/) (2025)
- [SAS-Prompt: Large Language Models as Numerical Optimizers for Robot Self-Improvement](https://mlsanity.com/paper/2504.20459/) (2025)

## Abstract (arXiv)

This paper addresses navigation by composite heterogeneous robots in a decentralized system when policy reasoning and local control operate at different update levels. In a NetLogo--Python implementation, three robots share motion dynamics but use different LLM backends. Each robot independently combines a large language model (LLM) policy agent, an Upper Confidence Bound (UCB) bandit, and a Double Deep Q-Network (Double DQN) controller; no central LLM generates team actions. LLM inference is confined to round-level policy generation and refinement rather than tick-level action selection. The robots perform cross-LLM communication through a shared round summary containing policies, outcomes, and learning feedback. UCB performs refinement-mode selection, and the policy-conditioned Double DQN performs tick-level action selection from navigation variables, active policy parameters, and the LLM action prior. Each of the four configurations was evaluated over 30 rounds. In the fixed simulation, the complete configuration reached the goal in all 90 correlated robot--round records and achieved the lowest median completion time (42 ticks) and P90 (73.2 ticks); its median was 25.0--39.1% lower than those of the other configurations. These observations provide descriptive, configuration-level evidence from the evaluated configurations.