mlsanityData to 7 Oct 2026

Few-Shot Bayesian Imitation Learning with Logical Program Policies

arXiv 1904.06317 ↗First posted 12 Apr 2019cs.AIcited by 30

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Humans can learn many novel tasks from a very small number (1--5) of demonstrations, in stark contrast to the data requirements of nearly tabula rasa deep learning methods. We propose an expressive class of policies, a strong but general prior, and a learning algorithm that, together, can learn interesting policies from very few examples. We represent policies as logical combinations of programs drawn from a domain-specific language (DSL), define a prior over policies with a probabilistic grammar, and derive an approximate Bayesian inference algorithm to learn policies from demonstrations. In experiments, we study five strategy games played on a 2D grid with one shared DSL. After a few demonstrations of each game, the inferred policies generalize to new game instances that differ substantially from the demonstrations. Our policy learning is 20--1,000x more data efficient than convolutional and fully convolutional policy learning and many orders of magnitude more computationally efficient than vanilla program induction. We argue that the proposed method is an apt choice for tasks that have scarce training data and feature significant, structured variation between task instances.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 13 of its 48 references that are in the map. First posted on arXiv 12 Apr 2019.

Builds on

13 of 48 references.

References · 13

  1. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  2. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
  3. 2017One-Shot Imitation Learningcited by 232
  4. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 225
  5. 2018Programmatically Interpretable Reinforcement Learningcited by 192
  6. 2016Neuro-Symbolic Program Synthesiscited by 84
  7. 2018Leveraging Grammar and Reinforcement Learning for Neural Program Synthesiscited by 83
  8. 2017Neural Task Programming: Learning to Generalize Across Hierarchical Taskscited by 75
  9. 2017Learning to Infer Graphics Programs from Hand-Drawn Imagescited by 73
  10. 2018Neural Task Graphs: Generalizing to Unseen Tasks from a Single Video Demonstrationcited by 60
  11. 2017Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physicscited by 57
  12. 2017DDCO: Discovery of Deep Continuous Options for Robot Learning from Demonstrationscited by 43
1 more
  1. 2018Beyond imitation: Zero-shot task transfer on robots by learning concepts as cognitive programscited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 2; 2023: 7; 2024: 8; 2025: 3; 2026: 1.

30 papers.

Sort:

By area:

  1. 2021Learning to Synthesize Programs as Interpretable and Generalizable Policiescited by 59
  2. 2021Leveraging Language to Learn Program Abstractions and Search Heuristicscited by 44
  3. 2021AGENT: A Benchmark for Core Psychological Reasoningcited by 42
  4. 2024Generating Code World Models with Large Language Models Guided by Monte Carlo Tree Searchcited by 38
  5. 2023Explainable Deep Reinforcement Learning: State of the Art and Challengescited by 33
  6. 2023Hierarchical Programmatic Reinforcement Learning via Learning to Compose Programscited by 20
  7. 2024Synthesizing Programmatic Reinforcement Learning Policies with Large Language Model Guided Searchcited by 13
  8. 2022Learning Sketches for Decomposing Planning Problems into Subproblems of Bounded Width: Extended Versioncited by 12
  9. 2023Learning Differentiable Logic Programs for Abstract Visual Reasoningcited by 12
  10. 2022Learning Neuro-symbolic Programs for Language Guided Robot Manipulationcited by 9
  11. 2023Efficient Symbolic Policy Learning with Differentiable Symbolic Expressioncited by 9
  12. 2021Learning General Policies from Small Examples Without Supervisioncited by 7
18 more
  1. 2023Hierarchical Neural Program Synthesiscited by 7
  2. 2023Generalized Planning as Heuristic Search: A new planning search-space that leverages pointers over objectscited by 6
  3. 2024Relational decomposition for program synthesiscited by 6
  4. 2021Target Languages (vs. Inductive Biases) for Learning to Act and Plancited by 5
  5. 2023Learning of Generalizable and Interpretable Knowledge in Grid-Based Reinforcement Learning Environmentscited by 5
  6. 2024From Code to Play: Benchmarking Program Search for Games Using Large Language Modelscited by 5
  7. 2021Stage Conscious Attention Network (SCAN) : A Demonstration-Conditioned Policy for Few-Shot Imitationcited by 2
  8. 2024KnowPC: Knowledge-Driven Programmatic Reinforcement Learning for Zero-shot Coordinationcited by 2
  9. 2021Supervised Bayesian Specification Inference from Demonstrationscited by 1
  10. 2021Automatic Discovery and Description of Human Planning Strategiescited by 1
  11. 2024Unveiling the Decision-Making Process in Reinforcement Learning with Genetic Programmingcited by 1
  12. 2021Learning logic programs by explaining their failurescited by 0
  13. 2024Guided Sketch-Based Program Induction by Search Gradientscited by 0
  14. 2024Sketch-Plan-Generalize: Learning and Planning with Neuro-Symbolic Programmatic Representations for Inductive Spatial Conceptscited by 0
  15. 2025An Empirical Comparison of Cost Functions in Inductive Logic Programmingcited by 0
  16. 2025A Unified Theory of Compositionality, Modularity, and Interpretability in Markov Decision Processescited by 0
  17. 2025Finite Automata Extraction: Low-data World Model Learning as Programs from Gameplay Videocited by 0
  18. 2026DiPRL: Learning Discrete Programmatic Policies via Architecture Entropy Regularizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A Bayesian Solution To The Imitation Gap
  2. 2020Augmenting Policy Learning with Routines Discovered from a Single Demonstration
  3. 2022A Ranking Game for Imitation Learning
  4. 2021Of Moments and Matching: A Game-Theoretic Framework for Closing the Imitation Gap
  5. 2022Semi-Supervised Imitation Learning of Team Policies from Suboptimal Demonstrations
  6. 2019Hierarchical Variational Imitation Learning of Control Programs
  7. 2018Fast Policy Learning through Imitation and Reinforcement
  8. 2020Safe Imitation Learning via Fast Bayesian Reward Inference from Preferences