Failure Modes of Deep Multi-Agent RL in Asynchronous Pricing: Reproducible Triggers, Trace Diagnostics, and a Partial Fix
Not on the map. Why?
Abstract (arXiv)
We study two reproducible failure modes of deep multi-agent reinforcement learning in continuous-time pricing markets: (i) tacit cartel formation between competing DDPG agents, and (ii) actor--critic instability at high event rates. We instantiate both inside a single CT-MARL benchmark (Poisson-clocked price updates, observation latency δ, interior-optimum logit demand), show that synchronous DDPG agents reliably trigger Failure Mode 1 with collusion index Delta = 0.69 ± 0.11, and quantify a partial microstructure fix: asynchrony alone cuts collusion by 48% and adding latency drives it to a minimum of Delta = 0.28. The fix has clearly documented costs: it is partial (Delta remains supra-Bertrand), it is non-monotone in δ, and it does not survive Failure Mode 2, which emerges as DDPG critic divergence at λ = 5 and corrupts the phase-diagram cell at (λ=5, δ=1). We accompany the scalar collusion index with trajectory-level trace diagnostics that expose the within-episode signalling collapse and the post-shock non-recovery.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 7 of its 13 references that are in the map. First posted on arXiv 3 Jun 2026.
Builds on
7 of 13 references.
References · 7
- 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
- 2020Model-based Reinforcement Learning for Semi-Markov Decision Processes with Neural ODEscited by 32
- 2025Algorithmic Pricing and Algorithmic Collusioncited by 3
- 2025Continuous-Time Value Iteration for Multi-Agent Reinforcement Learningcited by 2
- 2024Tacit algorithmic collusion in deep reinforcement learning guided price competition: A study using EV charge pricing gamecited by 1
- 2024By Fair Means or Foul: Quantifying Collusion in a Market Simulation with Deep Reinforcement Learningcited by 1
- 2025Impact of Price Inflation on Algorithmic Collusion Through Reinforcement Learning Agentscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Market-Alignment Risk in Pricing Agents: Trace Diagnostics and Trace-Prior RL under Hidden Competitor State
- 2026On the Fragility of AI Agent Collusion
- 2025How Market Volatility Shapes Algorithmic Collusion: A Comparative Analysis of Learning-Based Pricing Algorithms
- 2026Memory-Induced Supra-Competitive Outcomes Between Deep Reinforcement Learning Agents in Optimal Trade Execution
- 2026Zero Collapse: A Failure Mode of Policy Gradient Methods in Discontinuous Reward Environments
- 2026Learning from the Test: Self-Referential Differential Testing for Deep RL Agents
- 2022Calibration of Derivative Pricing Models: a Multi-Agent Reinforcement Learning Perspective
- 2025Game Theory and Multi-Agent Reinforcement Learning for Zonal Ancillary Markets ↗