mlsanityData to 14 Oct 2026

Failure Modes of Deep Multi-Agent RL in Asynchronous Pricing: Reproducible Triggers, Trace Diagnostics, and a Partial Fix

arXiv 2606.09884 ↗First posted 3 Jun 2026cs.MA cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We study two reproducible failure modes of deep multi-agent reinforcement learning in continuous-time pricing markets: (i) tacit cartel formation between competing DDPG agents, and (ii) actor--critic instability at high event rates. We instantiate both inside a single CT-MARL benchmark (Poisson-clocked price updates, observation latency δ, interior-optimum logit demand), show that synchronous DDPG agents reliably trigger Failure Mode 1 with collusion index Delta = 0.69 ± 0.11, and quantify a partial microstructure fix: asynchrony alone cuts collusion by 48% and adding latency drives it to a minimum of Delta = 0.28. The fix has clearly documented costs: it is partial (Delta remains supra-Bertrand), it is non-monotone in δ, and it does not survive Failure Mode 2, which emerges as DDPG critic divergence at λ = 5 and corrupts the phase-diagram cell at (λ=5, δ=1). We accompany the scalar collusion index with trajectory-level trace diagnostics that expose the within-episode signalling collapse and the post-shock non-recovery.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 7 of its 13 references that are in the map. First posted on arXiv 3 Jun 2026.

Builds on

7 of 13 references.

References · 7

  1. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  2. 2020Model-based Reinforcement Learning for Semi-Markov Decision Processes with Neural ODEscited by 32
  3. 2025Algorithmic Pricing and Algorithmic Collusioncited by 3
  4. 2025Continuous-Time Value Iteration for Multi-Agent Reinforcement Learningcited by 2
  5. 2024Tacit algorithmic collusion in deep reinforcement learning guided price competition: A study using EV charge pricing gamecited by 1
  6. 2024By Fair Means or Foul: Quantifying Collusion in a Market Simulation with Deep Reinforcement Learningcited by 1
  7. 2025Impact of Price Inflation on Algorithmic Collusion Through Reinforcement Learning Agentscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Market-Alignment Risk in Pricing Agents: Trace Diagnostics and Trace-Prior RL under Hidden Competitor State
  2. 2026On the Fragility of AI Agent Collusion
  3. 2025How Market Volatility Shapes Algorithmic Collusion: A Comparative Analysis of Learning-Based Pricing Algorithms
  4. 2026Memory-Induced Supra-Competitive Outcomes Between Deep Reinforcement Learning Agents in Optimal Trade Execution
  5. 2026Zero Collapse: A Failure Mode of Policy Gradient Methods in Discontinuous Reward Environments
  6. 2026Learning from the Test: Self-Referential Differential Testing for Deep RL Agents
  7. 2022Calibration of Derivative Pricing Models: a Multi-Agent Reinforcement Learning Perspective
  8. 2025Game Theory and Multi-Agent Reinforcement Learning for Zonal Ancillary Markets ↗