mlsanityData to 7 Oct 2026

The Medkit-Learn(ing) Environment: Medical Decision Modelling through Simulation

arXiv 2106.04240 ↗First posted 8 Jun 2021cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Understanding decision-making in clinical environments is of paramount importance if we are to bring the strengths of machine learning to ultimately improve patient outcomes. Several factors including the availability of public data, the intrinsically offline nature of the problem, and the complexity of human decision making, has meant that the mainstream development of algorithms is often geared towards optimal performance in tasks that do not necessarily translate well into the medical regime; often overlooking more niche issues commonly associated with the area. We therefore present a new benchmarking suite designed specifically for medical sequential decision making: the Medkit-Learn(ing) Environment, a publicly available Python package providing simple and easy access to high-fidelity synthetic medical data. While providing a standardised way to compare algorithms in a realistic medical setting we employ a generating process that disentangles the policy and environment dynamics to allow for a range of customisations, thus enabling systematic evaluation of algorithms' robustness against specific challenges prevalent in healthcare.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 27 of its 70 references that are in the map. First posted on arXiv 8 Jun 2021.

Builds on

27 of 70 references.

References · 27

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2013Auto-Encoding Variational Bayescited by 3,980
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  5. 2016Density estimation using Real NVPcited by 2,029
  6. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  7. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  8. 2016OpenAI Gymcited by 1,684
  9. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
  10. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  11. 2017Real-valued (Medical) Time Series Generation with Recurrent Conditional GANscited by 329
  12. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 256
15 more
  1. 2016Structured Inference Networks for Nonlinear State Space Modelscited by 182
  2. 2019Imitation Learning via Off-Policy Distribution Matchingcited by 182
  3. 2020Estimating Counterfactual Treatment Outcomes over Time Through Adversarially Balanced Representationscited by 154
  4. 2019Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Modelscited by 114
  5. 2020Online Bayesian Goal Inference for Boundedly-Rational Planning Agentscited by 67
  6. 2021Scalable Bayesian Inverse Reinforcement Learningcited by 56
  7. 2020Strictly Batch Imitation Learning by Energy-based Distribution Matchingcited by 54
  8. 2020Interpretable Off-Policy Evaluation in Reinforcement Learning by Highlighting Influential Transitionscited by 41
  9. 2020POPCORN: Partially Observed Prediction COnstrained ReiNforcement Learningcited by 36
  10. 2020Model-based Reinforcement Learning for Semi-Markov Decision Processes with Neural ODEscited by 32
  11. 2020The MAGICAL Benchmark for Robust Imitationcited by 30
  12. 2016Personalized Risk Scoring for Critical Care Prognosis using Mixtures of Gaussian Processescited by 19
  13. 2020Inverse Active Sensing: Modeling and Understanding Timely Decision-Makingcited by 12
  14. 2019Simitate: A Hybrid Imitation Learning Benchmarkcited by 10
  15. 2020Modelling Agent Policies with Interpretable Imitation Learningcited by 6

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 4; 2023: 3; 2024: 2; 2025: 1.

11 papers.

  1. 2023Can You Rely on Your Model Evaluation? Improving Model Evaluation with Synthetic Test Datacited by 31
  2. 2025Language Agents as Digital Representatives in Collective Decision-Makingcited by 15
  3. 2022POETREE: Interpretable Policy Learning with Adaptive Decision Treescited by 14
  4. 2023Accountability in Offline Reinforcement Learning: Explaining Decisions with a Corpus of Examplescited by 9
  5. 2022Inverse Online Learning: Understanding Non-Stationary and Reactionary Policiescited by 6
  6. 2022Practical Approaches for Fair Learning with Multitype and Multivariate Sensitive Attributescited by 3
  7. 2022Synthetic Model Combination: An Instance-wise Approach to Unsupervised Ensemble Learningcited by 2
  8. 2024OCMDP: Observation-Constrained Markov Decision Processcited by 2
  9. 2023Optimising Human-AI Collaboration by Learning Convincing Explanationscited by 1
  10. 2024IncomeSCM: From tabular data set to time-series simulator and causal estimation benchmarkcited by 1
  11. 2021ADCB: An Alzheimer's disease benchmark for evaluating observational estimators of causal effectscited by 0