mlsanityData to 7 Oct 2026

PufferLib: Making Reinforcement Learning Libraries and Environments Play Nice

arXiv 2406.12905 ↗First posted 11 Jun 2024cs.LG cs.AI cs.MAcited by 22

Not on the map. Why?

Abstract (arXiv)

You have an environment, a model, and a reinforcement learning library that are designed to work together but don't. PufferLib makes them play nice. The library provides one-line environment wrappers that eliminate common compatibility problems and fast vectorization to accelerate training. With PufferLib, you can use familiar libraries like CleanRL and SB3 to scale from classic benchmarks like Atari and Procgen to complex simulators like NetHack and Neural MMO. We release pip packages and prebuilt images with dependencies for dozens of environments. All of our code is free and open-source software under the MIT license, complete with baselines, documentation, and support at pufferai.github.io.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 10 of its 13 references that are in the map. First posted on arXiv 11 Jun 2024.

Builds on

10 of 13 references.

References · 10

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  3. 2016OpenAI Gymcited by 1,684
  4. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  5. 2020PettingZoo: Gym for Multi-Agent Reinforcement Learningcited by 209
  6. 2020The NetHack Learning Environmentcited by 196
  7. 2021Tianshou: a Highly Modularized Deep Reinforcement Learning Librarycited by 139
  8. 2020Sample Factory: Egocentric 3D Control from Pixels at 100000 FPS with Asynchronous Reinforcement Learningcited by 84
  9. 2022EnvPool: A Highly Parallel Reinforcement Learning Environment Execution Enginecited by 58
  10. 2021The Neural MMO Platform for Massively Multiagent Researchcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 8; 2026: 11.

22 papers.

Sort:

By area:

  1. 2024GPUDrive: Data-driven, multi-agent driving simulation at 1 million FPScited by 43
  2. 2024Gymnasium: A Standard Interface for Reinforcement Learning Environmentscited by 35
  3. 2025Scalable Option Learning in High-Throughput Environmentscited by 5
  4. 2026Human-like autonomy emerges from self-play and a pinch of human datacited by 4
  5. 2026Scaling Self-Play for End-to-End Drivingcited by 4
  6. 2026TerraTransfer: Learning End-to-End Driving Policies Without Expert Demonstrationscited by 3
  7. 2024Syllabus: Portable Curricula for Reinforcement Learning Agentscited by 2
  8. 2025Combining Pre-Trained Models for Enhanced Feature Representation in Reinforcement Learningcited by 2
  9. 2025CAMAR: Continuous Actions Multi-Agent Routingcited by 2
  10. 2025Multi-Agent Craftax: Benchmarking Open-Ended Multi-Agent Reinforcement Learning at the Hyperscalecited by 2
  11. 2026Equivariant Reinforcement Learning for Clifford Quantum Circuit Synthesiscited by 2
  12. 2026Pictura: Perspective-View Self-Play at Scale for Drivingcited by 2
10 more
  1. 2025Pokemon Red via Reinforcement Learningcited by 1
  2. 2025The challenge of hidden gifts in multi-agent reinforcement learningcited by 1
  3. 2026TerraZero: Procedural Driving Simulation for Zero-Demonstration Self-Play at Scalecited by 1
  4. 2025CiRL: Open-Source Environments for Reinforcement Learning in Circular Economy and Net Zerocited by 0
  5. 2025Assistax: A Multi-Agent Hardware-Accelerated Reinforcement Learning Benchmark for Assistive Roboticscited by 0
  6. 2026Automatic Generation of High-Performance RL Environmentscited by 0
  7. 2026A High-Throughput Compute-Efficient POMDP Hide-And-Seek-Engine (HASE) for Multi-Agent Operationscited by 0
  8. 2026PlayTrain: An Efficient Reinforcement Learning Framework for LLM-Generated Adaptable JavaScript Gamescited by 0
  9. 2026Comfort by Construction: Adaptive, Comfort-Bounded Action Spaces for Learned Driving Policiescited by 0
  10. 2026VISTA: An Attention-Based Multi-Agent Reinforcement Learning Architecture for Space Situational Awareness Sensor Taskingcited by 0