mlsanityData to 14 Oct 2026

Memory-Induced Supra-Competitive Outcomes Between Deep Reinforcement Learning Agents in Optimal Trade Execution

arXiv 2605.20348 ↗First posted 19 May 2026q-fin.CP cs.AIcited by 2

On the map: Deep reinforcement learning methods → Adversarial robustness and RL in finance

Abstract (arXiv)

In this paper, we investigate whether deep reinforcement-learning agents interacting in a shared optimal-execution environment can sustain supra-competitive outcomes, in the sense of achieving lower implementation shortfalls than the relevant game-theoretical competitive benchmark. We study a two-agent Almgren-Chriss liquidation game and examine how learned behavior depends on intra-episode environment feedback, the ability to interpret the mid-price and the agent's knoledge of the past. We first use ex-ante schedule-learning agents to remove intra-episode feedback and isolate what can arise when agents commit to complete liquidation trajectories before execution begins. We then allow agents to condition on the evolving state using a variety of DDQN architectures. We find that, when agents are given access to intra-episode history, especially recent prices and own past actions, supra-competitive outcomes become substantially more frequent and more persistent. These findings indicate that supra-competitive behavior in this execution game is driven not by multi-agent learning or by current price observation alone, but by feedback, memory, and state-contingent interaction along the realized execution path.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods. Builds on 7 of its 27 references that are in the map. First posted on arXiv 19 May 2026.

Builds on

7 of 27 references.

References · 7

  1. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  2. 2021Recent Advances in Reinforcement Learning in Financecited by 110
  3. 2018Double Deep Q-Learning for Optimal Executioncited by 43
  4. 2024A Pontryagin Perspective on Reinforcement Learningcited by 7
  5. 2013A state-constrained differential game arising in optimal portfolio liquidationcited by 3
  6. 2024Memoryless Algorithmic Collusion: Sure to Fail, Slow to Fallcited by 3
  7. 2025Algorithmic Collusion is Algorithm Orchestrationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Beyond Supra-Competitive Outcomes: Collusive Behaviour in Deep Reinforcement Learning for Optimal Execution Gamescited by 0
  2. 2026When a Correct Reward Is Not Enough: Diagnosing and Guiding PPO in an Analytically Solved Broker-Trader Gamecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Deviations from the Nash equilibrium in a two-player optimal execution game with reinforcement learning
  2. 2025Reinforcement Learning in Queue-Reactive Models: Application to Optimal Execution
  3. 2025Learning to Charge More: A Theoretical Study of Collusion by Q-Learning Agents
  4. 2025ABIDES-MARL: A Multi-Agent Reinforcement Learning Environment for Optimal Execution with Endogenous Liquidity ↗
  5. 2026SidConArena: An Environment Evaluating Agents in Open-Ended,Positive-Sum Bargaining Game
  6. 2025Dueling Deep Reinforcement Learning for Financial Time Series
  7. 2024Optimal Execution with Reinforcement Learning
  8. 2026Strategic Bargaining in Multi-Buyer Markets: Reinforcement Learning from Verifiable Rewards for LLM Negotiations