pix2pockets: Shot Suggestions in 8-Ball Pool from a Single Image in the Wild
On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation
Abstract (arXiv)
Computer vision models have seen increased usage in sports, and reinforcement learning (RL) is famous for beating humans in strategic games such as Chess and Go. In this paper, we are interested in building upon these advances and examining the game of classic 8-ball pool. We introduce pix2pockets, a foundation for an RL-assisted pool coach. Given a single image of a pool table, we first aim to detect the table and the balls and then propose the optimal shot suggestion. For the first task, we build a dataset with 195 diverse images where we manually annotate all balls and table dots, leading to 5748 object segmentation masks. For the second task, we build a standardized RL environment that allows easy development and benchmarking of any RL algorithm. Our object detection model yields an AP50 of 91.2 while our ball location pipeline obtains an error of only 0.4 cm. Furthermore, we compare standard RL algorithms to set a baseline for the shot suggestion task and we show that all of them fail to pocket all balls without making a foul move. We also present a simple baseline that achieves a per-shot success rate of 94.7% and clears a full game in a single turn 30% of the time.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 18 of its 38 references that are in the map. First posted on arXiv 16 Apr 2025.
Builds on
18 of 38 references.
References · 18
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
- 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
- 2015Continuous control with deep reinforcement learningcited by 3,407
- 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
- 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
- 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
- 2016OpenAI Gymcited by 1,736
- 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 734
- 2020A Closer Look at Invalid Action Masking in Policy Gradient Algorithmscited by 171
- 2018SoccerNet: A Scalable Dataset for Action Spotting in Soccer Videoscited by 150
- 2016Control of Memory, Active Perception, and Action in Minecraftcited by 61
6 more
- 2021Counter-Strike Deathmatch with Large-Scale Behavioural Cloningcited by 50
- 2020Game Plan: What AI can do for Football, and What Football can do for AIcited by 27
- 2015Detecting events and key actors in multi-person videoscited by 26
- 2020Cracking the Black Box: Distilling Deep Sports Analyticscited by 12
- 2024Improving Object Detection Quality in Football Through Super-Resolution Techniquescited by 6
- 2018Deep Reinforcement Learning with Model Learning and Monte Carlo Tree Search in Minecraftcited by 5
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2024CNN-based Game State Detection for a Foosball Table
- 2026BilliardPhys-Bench: Benchmarking Physical Reasoning and Visual Dynamics of Multimodal LLMs
- 2024SkillMimic: Learning Basketball Interaction Skills from Demonstrations
- 2024Competing for pixels: a self-play algorithm for weakly-supervised segmentation
- 2026Learning to Throw Objects Safely in Multi-Obstacle Environments
- 2026ShuttleArena: Interpretable Self-Play in Physics-Based Badminton
- 2026Humanoid Badminton: Learning Dynamic Racket Skills from Limited Human Motion Data ↗
- 2026Maximising the Set-Piece Return: Optimising Football Corner Tactics with Graph Reinforcement Learning