When Outcome Looks Right But Discipline Fails: Trace-Based Evaluation Under Hidden Competitor State
Not on the map. Why?
Abstract (arXiv)
Outcome-only evaluation can certify economically unsafe agents: a policy can hit a business KPI while violating deployable behavioral discipline. In hotel pricing with hidden competitor state, a learner can achieve plausible revenue per available room while failing to preserve the rate discipline of a rule-based revenue-management competitor. We introduce discipline stability, a trace-based evaluation paradigm: define the benchmark behavior, restrict observations to the deployment regime, induce trace diagnostics from failure, separate mechanisms with ablations, and test transfer and deployment. Across a two-hotel benchmark and a compact hidden-budget bidding task, reward-only PPO variants miss trace alignment; revealing hidden state reduces label uncertainty; deterministic copy collapses uncertainty; and trace-prior or corrected history policies better preserve price or bid distributions. Pure behavior cloning is nearly enough for symmetric imitation, while Trace-Prior RL adds bounded adaptation under capacity asymmetry. The contribution is an evaluation and benchmark paradigm, not a new optimizer or a universal claim about MARL
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 9 of its 24 references that are in the map. First posted on arXiv 18 May 2026.
Builds on
9 of 24 references.
References · 9
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
- 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
- 2019Behavior Regularized Offline Reinforcement Learningcited by 654
- 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 647
- 2018Maximum a Posteriori Policy Optimisationcited by 318
- 2022The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Modelscited by 312
- 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 283
- 2026Market-Alignment Risk in Pricing Agents: Trace Diagnostics and Trace-Prior RL under Hidden Competitor Statecited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026LiveMACE: Process-Aware Evaluation of LLM Agent Capabilities in Evolving Markets
- 2026Credit Without Ground Truth: Auditing Step-Level Credit Assignment in LLM Agents Against Executed Replay
- 2026When Aggregate Alignment Misleads: Auditing Policy Repair Without Per-State Expert Actions
- 2026PriceBench: A Diagnostic Benchmark for Price, Quality, and Brand Preferences in LLM Booking Agents
- 2026Success Conditioning as Policy Improvement: The Optimization Problem Solved by Imitating Success
- 2026STOCKTAKE: Measuring the Gap Between Perception and Action in LLM Agents with a Fair Oracle
- 2026Trace2Policy: From Expert Behavior Traces to Self-Evolving Decision Agents
- 2026TRACE: Turn-level Reward Assignment via Credit Estimation for Long-Horizon Agents