mlsanityData to 7 Oct 2026

WFA-IRL: Inverse Reinforcement Learning of Autonomous Behaviors Encoded as Weighted Finite Automata

arXiv 2103.05895 ↗First posted 10 Mar 2021cs.LG cs.LOcited by 0

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

This paper presents a method for learning logical task specifications and cost functions from demonstrations. Constructing specifications by hand is challenging for complex objectives and constraints in autonomous systems. Instead, we consider demonstrated task executions, whose logic structure and transition costs need to be inferred by an autonomous agent. We employ a spectral learning approach to extract a weighted finite automaton (WFA), approximating the unknown task logic. Thereafter, we define a product between the WFA for high-level task guidance and a labeled Markov decision process for low-level control. An inverse reinforcement learning (IRL) problem is considered to learn a cost function by backpropagating the loss between agent and expert behaviors through the planning algorithm. Our proposed model, termed WFA-IRL, is capable of generalizing the execution of the inferred task specification in a suite of MiniGrid environments.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 24 of its 60 references that are in the map. First posted on arXiv 10 Mar 2021.

Builds on

24 of 60 references.

Closest ancestors · 1

Same direction, newest first

  1. 2020Learning Navigation Costs from Demonstration in Partially Observable Environments26 Feb

Other references · 23

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2016Generative Adversarial Imitation Learningcited by 1,576
  5. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 457
  6. 2016The Option-Critic Architecturecited by 433
  7. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  8. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 282
  9. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learningcited by 203
  10. 2016Learning to Navigate in Complex Environmentscited by 192
  11. 2019Reinforcement Learning for Temporal Logic Control Synthesis with Probabilistic Satisfaction Guaranteescited by 77
  12. 2017Multi-Level Discovery of Deep Optionscited by 50
11 more
  1. 2020Deep Imitation Learning for Bimanual Robotic Manipulationcited by 49
  2. 2021Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logiccited by 38
  3. 2016Watch This: Scalable Cost-Function Learning for Path Planning in Urban Environmentscited by 33
  4. 2018Learning Abstract Optionscited by 31
  5. 2019Modular Deep Reinforcement Learning with Temporal Logic Specificationscited by 25
  6. 2018Connecting Weighted Automata and Recurrent Neural Networks through Spectral Learningcited by 15
  7. 2020Elaborating on Learned Demonstrations with Temporal Logic Specificationscited by 14
  8. 2020Explaining Multi-stage Tasks by Learning Temporal Logic Formulas from Suboptimal Demonstrationscited by 13
  9. 2018Formal Specification and Verification of Autonomous Robotic Systems: A Surveycited by 9
  10. 2015Optimal Temporal Logic Planning in Probabilistic Semantic Mapscited by 3
  11. 2018Robotics and Integrated Formal Methods: Necessity meets Opportunitycited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2021Inverse Reinforcement Learning: A Control Lyapunov Approach
  2. 2024Stable Inverse Reinforcement Learning: Policies from Control Lyapunov Landscapes
  3. 2021Task-Guided Inverse Reinforcement Learning Under Partial Information
  4. 2025Inverse Reinforcement Learning via Convex Optimization
  5. 2019CWAE-IRL: Formulating a supervised approach to Inverse Reinforcement Learning problem
  6. 2021Inverse Reinforcement Learning in a Continuous State Space with Formal Guarantees
  7. 2022Reward Learning using Structural Motifs in Inverse Reinforcement Learning
  8. 2022Task-Guided IRL in POMDPs that Scales