# Prompt Optimization Enables Stable Algorithmic Collusion in LLM Agents

arXiv: https://arxiv.org/abs/2604.17774 · HTML: https://mlsanity.com/paper/2604.17774/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Financial trading and forecasting agents, one of 16 directions in LLM agents and tool use.
- Builds on 11 of its 22 references that are in the map.
- First posted on arXiv 20 Apr 2026.

## Placed in

- LLM agents and tool use / [Financial trading and forecasting agents](https://mlsanity.com/agents/financial-trading-and-forecasting-agents/)

## Builds on (most cited first, 11 of 11)

- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback](https://mlsanity.com/paper/2309.00267/) (2023)
- [Secret Collusion among AI Agents: Multi-Agent Deception via Steganography](https://mlsanity.com/paper/2402.07510/) (2024)
- [Algorithmic Collusion by Large Language Models](https://mlsanity.com/paper/2404.00806/) (2024)
- [Strategic Collusion of LLM Agents: Market Division in Multi-Commodity Competitions](https://mlsanity.com/paper/2410.00031/) (2024)
- [Distributional AGI Safety](https://mlsanity.com/paper/2512.16856/) (2025)
- [EconEvals: Benchmarks and Litmus Tests for Economic Decision-Making by LLM Agents](https://mlsanity.com/paper/2503.18825/) (2025)
- [On the Fragility of AI Agent Collusion](https://mlsanity.com/paper/2603.20281/) (2026)
- [ZERA: Zero-init Instruction Evolving Refinement Agent -- From Zero Instructions to Structured Prompts via Principle-based Optimization](https://mlsanity.com/paper/2509.18158/) (2025)
- [Aligning Large Language Model Agents with Rational and Moral Preferences: A Supervised Fine-Tuning Approach](https://mlsanity.com/paper/2507.20796/) (2025)

## Built on by (most cited first, 4 of 4)

- [Emergent Collusion in Long-Horizon LLM Agent Interaction](https://mlsanity.com/paper/2609.24967/) (2026)
- [SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systems](https://mlsanity.com/paper/2609.00595/) (2026)
- [From Economic Agents to Agentic Economies: A Systems Blueprint for Economic World Models](https://mlsanity.com/paper/2608.06020/) (2026)
- [Persistent Partners Raise Prices Among Learning Agents](https://mlsanity.com/paper/2609.35402/) (2026)

## Abstract (arXiv)

LLM agents in markets present algorithmic collusion risks. While prior work shows LLM agents reach supracompetitive prices through tacit coordination, existing research focuses on hand-crafted prompts. The emerging paradigm of prompt optimization necessitates new methodologies for understanding autonomous agent behavior. We investigate whether prompt optimization leads to emergent collusive behaviors in market simulations. We propose a meta-learning loop where LLM agents participate in duopoly markets and an LLM meta-optimizer iteratively refines shared strategic guidance. Our experiments reveal that meta-prompt optimization enables agents to discover stable tacit collusion strategies with substantially improved coordination quality compared to baseline agents. These behaviors generalize to held-out test markets, indicating discovery of general coordination principles. Analysis of evolved prompts reveals systematic coordination mechanisms through stable shared strategies. Our findings call for further investigation into AI safety implications in autonomous multi-agent systems.