mlsanityData to 7 Oct 2026

Horizon: Facebook's Open Source Applied Reinforcement Learning Platform

arXiv 1811.00260 ↗First posted 1 Nov 2018cs.LG cs.AI stat.MLcited by 48

Not on the map. Why?

Abstract (arXiv)

In this paper we present Horizon, Facebook's open source applied reinforcement learning (RL) platform. Horizon is an end-to-end platform designed to solve industry applied RL problems where datasets are large (millions to billions of observations), the feedback loop is slow (vs. a simulator), and experiments must be done with care because they don't run in a simulator. Unlike other RL platforms, which are often designed for fast prototyping and experimentation, Horizon is designed with production use cases as top of mind. The platform contains workflows to train popular deep RL algorithms and includes data preprocessing, feature transformation, distributed training, counterfactual policy evaluation, optimized serving, and a model-based data understanding tool. We also showcase and describe real examples where reinforcement learning models trained with Horizon significantly outperformed and replaced supervised learning systems at Facebook.

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Nov 2018.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2019: 1; 2020: 1; 2021: 19; 2022: 8; 2023: 11; 2024: 7; 2025: 1.

48 papers.

Sort:

By area:

  1. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  2. 2023Harms from Increasingly Agentic Algorithmic Systemscited by 134
  3. 2022CORL: Research-oriented Deep Offline Reinforcement Learning Librarycited by 117
  4. 2021Truthful AI: Developing and governing AI that does not liecited by 109
  5. 2019Benchmarking Batch Deep Reinforcement Learning Algorithmscited by 108
  6. 2023TorchRL: A data-driven decision-making library for PyTorchcited by 59
  7. 2024AI Alignment with Changing and Influenceable Reward Functionscited by 33
  8. 2022Estimating and Penalizing Induced Preference Shifts in Recommender Systemscited by 29
  9. 2021Showing Your Offline Reinforcement Learning Work: Online Evaluation Budget Matterscited by 24
  10. 2021User Tampering in Reinforcement Learning Recommender Systemscited by 23
  11. 2021RecSim NG: Toward Principled Uncertainty Modeling for Recommender Ecosystemscited by 21
  12. 2021A Deep Reinforcement Learning Approach to Marginalized Importance Sampling with the Successor Representationcited by 19
36 more
  1. 2021RLOps: Development Life-cycle of Reinforcement Learning Aided Open RANcited by 15
  2. 2023SRL: Scaling Distributed Reinforcement Learning to Over Ten Thousand Corescited by 10
  3. 2021Looper: An end-to-end ML platform for product decisionscited by 9
  4. 2023Pearl: A Production-ready Reinforcement Learning Agentcited by 9
  5. 2024Monitored Markov Decision Processescited by 9
  6. 2022Evaluation Beyond Task Performance: Analyzing Concepts in AlphaZero in Hexcited by 8
  7. 2024Acceleration for Deep Reinforcement Learning using Parallel and Distributed Computing: A Surveycited by 8
  8. 2023Modeling Recommender Ecosystems: Research Challenges at the Intersection of Mechanism Design, Reinforcement Learning and Generative Modelscited by 7
  9. 2023SCOPE-RL: A Python Library for Offline Reinforcement Learning and Off-Policy Evaluationcited by 6
  10. 2021RL4RS: A Real-World Dataset for Reinforcement Learning based Recommender Systemcited by 5
  11. 2022Rule Mining over Knowledge Graphs via Reinforcement Learningcited by 5
  12. 2022Reinforcement Learning in Practice: Opportunities and Challengescited by 5
  13. 2022Synthetic Data-Based Simulators for Recommender Systems: A Surveycited by 5
  14. 2023RetailSynth: Synthetic Data Generation for Retail AI Systems Evaluationcited by 5
  15. 2021Personalization for Web-based Services using Offline Reinforcement Learningcited by 3
  16. 2021Discovering an Aid Policy to Minimize Student Evasion Using Offline Reinforcement Learningcited by 3
  17. 2021Socialbots on Fire: Modeling Adversarial Behaviors of Socialbots via Multi-Agent Hierarchical Reinforcement Learningcited by 3
  18. 2023Human-Centric Resource Allocation for the Metaverse With Multiaccess Edge Computingcited by 3
  19. 2021Bellman: A Toolbox for Model-Based Reinforcement Learning in TensorFlowcited by 2
  20. 2020Distilled Thompson Sampling: Practical and Efficient Thompson Sampling via Imitation Learningcited by 1
  21. 2021Evaluating the progress of Deep Reinforcement Learning in the real world: aligning domain-agnostic and domain-specific researchcited by 1
  22. 2021Safer Reinforcement Learning through Transferable Instinct Networkscited by 1
  23. 2022Temporal Multimodal Multivariate Learningcited by 1
  24. 2022Karolos: An Open-Source Reinforcement Learning Framework for Robot-Task Environmentscited by 1
  25. 2023An Intentional Forgetting-Driven Self-Healing Method For Deep Reinforcement Learning Systemscited by 1
  26. 2024Knowledge Transfer for Cross-Domain Reinforcement Learning: A Systematic Reviewcited by 1
  27. 2024Neural Click Models for Recommender Systemscited by 1
  28. 2025Nuclear Microreactor Control with Deep Reinforcement Learningcited by 1
  29. 2021The AI Arena: A Framework for Distributed Multi-Agent Reinforcement Learningcited by 0
  30. 2021Techniques Toward Optimizing Viewability in RTB Ad Campaigns Using Reinforcement Learningcited by 0
  31. 2021GuideBoot: Guided Bootstrap for Deep Contextual Banditscited by 0
  32. 2021A Validation Tool for Designing Reinforcement Learning Environmentscited by 0
  33. 2023SmartChoices: Augmenting Software with Learned Implementationscited by 0
  34. 2023Representation Learning in Low-rank Slate-based Recommender Systemscited by 0
  35. 2024PerOS: Personalized Self-Adapting Operating Systems in the Cloudcited by 0
  36. 2024Minimal Batch Adaptive Learning Policy Engine for Real-Time Mid-Price Forecasting in High-Frequency Tradingcited by 0