mlsanityData to 7 Oct 2026

LIFT: Reinforcement Learning in Computer Systems by Learning From Demonstrations

arXiv 1808.07903 ↗First posted 23 Aug 2018cs.LG stat.MLcited by 8

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

Reinforcement learning approaches have long appealed to the data management community due to their ability to learn to control dynamic behavior from raw system performance. Recent successes in combining deep neural networks with reinforcement learning have sparked significant new interest in this domain. However, practical solutions remain elusive due to large training data requirements, algorithmic instability, and lack of standard tools. In this work, we introduce LIFT, an end-to-end software stack for applying deep reinforcement learning to data management tasks. While prior work has frequently explored applications in simulations, LIFT centers on utilizing human expertise to learn from demonstrations, thus lowering online training times. We further introduce TensorForce, a TensorFlow library for applied deep reinforcement learning exposing a unified declarative interface to common RL algorithms, thus providing a backend to LIFT. We demonstrate the utility of LIFT in two case studies in database compound indexing and resource management in stream processing. Results show LIFT controllers initialized from demonstrations can outperform human baselines and heuristics across latency metrics and space usage by up to 70%.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 29 of its 77 references that are in the map. First posted on arXiv 23 Aug 2018.

Builds on

29 of 77 references.

References · 29

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  5. 2015Trust Region Policy Optimizationcited by 2,503
  6. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  7. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  8. 2016Generative Adversarial Imitation Learningcited by 1,576
  9. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  10. 2017Deep Reinforcement Learning that Matterscited by 727
  11. 2016TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systemscited by 660
  12. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
17 more
  1. 2017A Brief Survey of Deep Reinforcement Learningcited by 466
  2. 2017Snorkel: Rapid Training Data Creation with Weak Supervisioncited by 370
  3. 2017Deep Reinforcement Learning: An Overviewcited by 327
  4. 2015MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systemscited by 195
  5. 2017The Case for Learned Index Structurescited by 188
  6. 2016Continuous Deep Q-Learning with Model-based Accelerationcited by 149
  7. 2017Robust Imitation of Diverse Behaviorscited by 85
  8. 2017Device Placement Optimization with Reinforcement Learningcited by 84
  9. 2018Simple random search provides a competitive approach to reinforcement learningcited by 75
  10. 2018Reinforcement Learning from Imperfect Demonstrationscited by 74
  11. 2017Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physicscited by 57
  12. 2017RLlib: Abstractions for Distributed Reinforcement Learningcited by 53
  13. 2017Domain Randomization and Generative Models for Robotic Graspingcited by 34
  14. 2017A Benchmark Environment Motivated by Industrial Control Problemscited by 31
  15. 2017Deep Q-learning from Demonstrationscited by 27
  16. 2018The Case for Automatic Database Administration using Deep Reinforcement Learningcited by 8
  17. 2017A Machine Learning Approach to Routingcited by 2

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 3; 2023: 1; 2024: 3.

8 papers.

  1. 2023Dynamic Feature-based Deep Reinforcement Learning for Flow Control of Circular Cylinder with Sparse Surface Pressure Sensingcited by 7
  2. 2024Advanced deep-reinforcement-learning methods for flow control: group-invariant and positional-encoding networks improve learning speed and qualitycited by 5
  3. 2021High-Dimensional Bayesian Optimization with Multi-Task Learning for RocksDBcited by 4
  4. 2022A Comparison of Reinforcement Learning Frameworks for Software Testing Taskscited by 3
  5. 2024A semi-centralized multi-agent RL framework for efficient irrigation schedulingcited by 2
  6. 2022Improving Post-Processing of Audio Event Detectors Using Reinforcement Learningcited by 1
  7. 2024Optimal Parallelization Strategies for Active Flow Control in Deep Reinforcement Learning-Based Computational Fluid Dynamicscited by 1
  8. 2022Multi-agent Databases via Independent Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Launchpad: Learning to Schedule Using Offline and Online RL Methods
  2. 2024D5RL: Diverse Datasets for Data-Driven Deep Reinforcement Learning
  3. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learning
  4. 2025DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Search
  5. 2022Large-Scale Retrieval for Reinforcement Learning
  6. 2021DCUR: Data Curriculum for Teaching via Samples with Reinforcement Learning
  7. 2025PyTupli: A Scalable Infrastructure for Collaborative Offline Reinforcement Learning Projects
  8. 2025DemoTuner: Automatic Performance Tuning for Database Management Systems Based on Demonstration Reinforcement Learning