# Improving Bidding and Playing Strategies in the Trick-Taking game Wizard using Deep Q-Networks

arXiv: https://arxiv.org/abs/2205.13834 · HTML: https://mlsanity.com/paper/2205.13834/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 18 references that are in the map.
- First posted on arXiv 27 May 2022.

## Builds on (most cited first, 5 of 5)

- [Deep Recurrent Q-Learning for Partially Observable MDPs](https://mlsanity.com/paper/1507.06527/) (2015)
- [Combining Deep Reinforcement Learning and Search for Imperfect-Information Games](https://mlsanity.com/paper/2007.13544/) (2020)
- [Automatic Bridge Bidding Using Deep Reinforcement Learning](https://mlsanity.com/paper/1607.03290/) (2016)
- [Improving Search with Supervised Learning in Trick-Based Card Games](https://mlsanity.com/paper/1903.09604/) (2019)
- [Learning Policies from Human Data for Skat](https://mlsanity.com/paper/1905.10907/) (2019)

## Abstract (arXiv)

In this work, the trick-taking game Wizard with a separate bidding and playing phase is modeled by two interleaved partially observable Markov decision processes (POMDP). Deep Q-Networks (DQN) are used to empower self-improving agents, which are capable of tackling the challenges of a highly non-stationary environment. To compare algorithms between each other, the accuracy between bid and trick count is monitored, which strongly correlates with the actual rewards and provides a well-defined upper and lower performance bound. The trained DQN agents achieve accuracies between 66% and 87% in self-play, leaving behind both a random baseline and a rule-based heuristic. The conducted analysis also reveals a strong information asymmetry concerning player positions during bidding. To overcome the missing Markov property of imperfect-information games, a long short-term memory (LSTM) network is implemented to integrate historic information into the decision-making process. Additionally, a forward-directed tree search is conducted by sampling a state of the environment and thereby turning the game into a perfect information setting. To our surprise, both approaches do not surpass the performance of the basic DQN agent.