# Learning to sample fibers for goodness-of-fit testing

arXiv: https://arxiv.org/abs/2405.13950 · HTML: https://mlsanity.com/paper/2405.13950/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 11 of its 62 references that are in the map.
- First posted on arXiv 22 May 2024.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 11 of 11)

- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizon](https://mlsanity.com/paper/1811.06128/) (2018)
- [Reinforcement Learning for Integer Programming: Learning to Cut](https://mlsanity.com/paper/1906.04859/) (2019)
- [Coauthorship and Citation Networks for Statisticians](https://mlsanity.com/paper/1410.2840/) (2014)
- [Convergence Proof for Actor-Critic Methods Applied to PPO and RUDDER](https://mlsanity.com/paper/2012.01399/) (2020)
- [Stability of Stochastic Approximations with `Controlled Markov' Noise and Temporal Difference Learning](https://mlsanity.com/paper/1504.06043/) (2015)
- [Turning Mathematics Problems into Games: Reinforcement Learning and Gr"obner bases together solve Integer Feasibility Problems](https://mlsanity.com/paper/2208.12191/) (2022)
- [Markov bases: a 25 year update](https://mlsanity.com/paper/2306.06270/) (2023)
- [Coauthorship and citation networks for statisticians: Comment](https://mlsanity.com/paper/1608.06667/) (2016)
- [What is... a Markov basis?](https://mlsanity.com/paper/1907.07320/) (2019)

## Abstract (arXiv)

We consider the problem of constructing exact goodness-of-fit tests for discrete exponential family models. This classical problem remains practically unsolved for many types of structured or sparse data, as it rests on a computationally difficult core task: to produce a reliable sample from lattice points in a high-dimensional polytope. We translate the problem into a Markov decision process and demonstrate a reinforcement learning approach for learning `good moves' for sampling. We illustrate the approach on data sets and models for which traditional MCMC samplers converge too slowly due to problem size, sparsity structure, and the requirement to use prohibitive non-linear algebra computations in the process. The differentiating factor is the use of scalable tools from linear algebra in the context of theoretical guarantees provided by non-linear algebra. Our algorithm is based on an actor-critic sampling scheme, with provable convergence. The discovered moves can be used to efficiently obtain an exchangeable sample, significantly cutting computational times with regards to statistical testing.