mlsanityData to 14 Oct 2026

SwiftRL: Towards Efficient Reinforcement Learning on Real Processing-In-Memory Systems

arXiv 2405.03967 ↗First posted 7 May 2024cs.LG cs.AI cs.ARcited by 2

On the map: Deep reinforcement learning methods → RL training of language and agent models

Abstract (arXiv)

Reinforcement Learning (RL) trains agents to learn optimal behavior by maximizing reward signals from experience datasets. However, RL training often faces memory limitations, leading to execution latencies and prolonged training times. To overcome this, SwiftRL explores Processing-In-Memory (PIM) architectures to accelerate RL workloads. We achieve near-linear performance scaling by implementing RL algorithms like Tabular Q-learning and SARSA on UPMEM PIM systems and optimizing for hardware. Our experiments on OpenAI GYM environments using UPMEM hardware demonstrate superior performance compared to CPU and GPU implementations.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods. Builds on 57 of its 149 references that are in the map. First posted on arXiv 7 May 2024.

Builds on

57 of 149 references.

References · 57

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  2. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  3. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  4. 2016OpenAI Gymcited by 1,736
  5. 2017Mixed Precision Trainingcited by 1,156
  6. 2019Solving Rubik's Cube with a Robot Handcited by 734
  7. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 721
  8. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 426
  9. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 402
  10. 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 311
  11. 2019Reinforcement Learning in Healthcare: A Surveycited by 310
  12. 2020Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?cited by 294
38 more
  1. 2017StarCraft II: A New Challenge for Reinforcement Learningcited by 274
  2. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 231
  3. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 181
  4. 2019Improving Neural Network Quantization without Retraining using Outlier Channel Splittingcited by 151
  5. 2020Acme: A Research Framework for Distributed Reinforcement Learningcited by 146
  6. 2020SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Drivingcited by 121
  7. 2020Sample Complexity of Asynchronous Q-Learning: Sharper Analysis and Variance Reductioncited by 95
  8. 2021Federated Reinforcement Learning: Techniques, Applications, and Open Challengescited by 81
  9. 2018Generative Adversarial User Model for Reinforcement Learning Based Recommendation Systemcited by 64
  10. 2019Multi-Agent Connected Autonomous Driving using Deep Reinforcement Learningcited by 53
  11. 2021DeepThermal: Combustion Optimization for Thermal Power Generating Units Using Offline Reinforcement Learningcited by 48
  12. 2023Leveraging Factored Action Spaces for Efficient Offline Reinforcement Learning in Healthcarecited by 42
  13. 2018Accelerated Methods for Deep Reinforcement Learningcited by 40
  14. 2023The Blessing of Heterogeneity in Federated Q-Learning: Linear Speedup and Beyondcited by 36
  15. 2017Efficient Parallel Methods for Deep Reinforcement Learningcited by 32
  16. 2020A Modern Primer on Processing in Memorycited by 32
  17. 2018CM3: Cooperative Multi-goal Multi-stage Multi-agent Reinforcement Learningcited by 31
  18. 2021Google Neural Network Models for Edge Devices: Analyzing and Mitigating Machine Learning Inference Bottleneckscited by 23
  19. 2021Independent Learning in Stochastic Gamescited by 22
  20. 2019Processing Data Where It Makes Sense: Enabling In-Memory Computationcited by 21
  21. 2023MAC-PO: Multi-Agent Experience Replay via Collective Priority Optimizationcited by 20
  22. 2019QuaRL: Quantization for Fast and Environmentally Sustainable Reinforcement Learningcited by 19
  23. 2021Pythia: A Customizable Hardware Prefetching Framework Using Online Reinforcement Learningcited by 18
  24. 2023Pathfinding Future PIM Architectures by Demystifying a Commercial PIM Technologycited by 14
  25. 2021Benchmarking Memory-Centric Computing Systems: Analysis of Real Processing-in-Memory Hardwarecited by 13
  26. 2022Sibyl: Adaptive and Extensible Data Placement in Hybrid Storage Systems Using Online Reinforcement Learningcited by 8
  27. 2023PIM-GPT: A Hybrid Process-in-Memory Accelerator for Autoregressive Transformerscited by 8
  28. 2024The Landscape of Compute-near-memory and Compute-in-memory: A Research and Commercial Overviewcited by 8
  29. 2023AccMER: Accelerating Multi-Agent Experience Replay with Cache Locality-aware Prioritizationcited by 7
  30. 2019Deep Q-Learning with Q-Matrix Transfer Learning for Novel Fire Evacuation Environmentcited by 6
  31. 2019In-DRAM Bulk Bitwise Execution Enginecited by 6
  32. 2023Scalability Bottlenecks in Multi-Agent Reinforcement Learning Systemscited by 6
  33. 2018A Scalable Near-Memory Architecture for Training Deep Neural Networks on Large In-Memory Datasetscited by 5
  34. 2022A Framework for High-throughput Sequence Alignment using Real Processing-in-Memory Systemscited by 5
  35. 2022Accelerating Neural Network Inference with Processing-in-DRAM: From the Edge to the Cloudcited by 5
  36. 2022CINM (Cinnamon): A Compilation Infrastructure for Heterogeneous Compute In-Memory and Compute Near-Memory Paradigmscited by 5
  37. 2023SimplePIM: A Software Framework for Productive and Efficient Processing-in-Memorycited by 5
  38. 2024PIM-Opt: Demystifying Distributed Optimization Algorithms on a Real-World Processing-In-Memory Systemcited by 5

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 0; 2026: 1.

2 papers.

  1. 2026LOCALUT: Harnessing Capacity-Computation Tradeoffs for LUT-Based Inference in DRAM-PIMcited by 1
  2. 2024UpANNS: Enhancing Billion-Scale ANNS Efficiency with Real-World PIM Architecturecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025LlamaRL: A Distributed Asynchronous Reinforcement Learning Framework for Efficient Large-scale LLM Training
  2. 2026AuroraRL: Fast, Fault-Tolerant, and Cost-Efficient Reinforcement Learning over Decentralized Network
  3. 2025The Art of Scaling Reinforcement Learning Compute for LLMs
  4. 2024Reinforcement Learning for Dynamic Memory Allocation
  5. 2026BitRL: Reinforcement Learning with 1-bit Quantized Language Models for Resource-Constrained Edge Deployment
  6. 2025PipelineRL: Faster On-policy Reinforcement Learning for Long Sequence Generation
  7. 2025TTRL: Test-Time Reinforcement Learning
  8. 2025AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning