mlsanityData to 14 Oct 2026

Inverse-Inverse Reinforcement Learning. How to Hide Strategy from an Adversarial Inverse Reinforcement Learner

arXiv 2205.10802 ↗First posted 22 May 2022cs.LG eess.SPcited by 3

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

Inverse reinforcement learning (IRL) deals with estimating an agent's utility function from its actions. In this paper, we consider how an agent can hide its strategy and mitigate an adversarial IRL attack; we call this inverse IRL (I-IRL). How should the decision maker choose its response to ensure a poor reconstruction of its strategy by an adversary performing IRL to estimate the agent's strategy? This paper comprises four results: First, we present an adversarial IRL algorithm that estimates the agent's strategy while controlling the agent's utility function. Our second result for I-IRL result spoofs the IRL algorithm used by the adversary. Our I-IRL results are based on revealed preference theory in micro-economics. The key idea is for the agent to deliberately choose sub-optimal responses that sufficiently masks its true strategy. Third, we give a sample complexity result for our main I-IRL result when the agent has noisy estimates of the adversary specified utility function. Finally, we illustrate our I-IRL scheme in a radar problem where a meta-cognitive radar is trying to mitigate an adversarial target.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 5 of its 32 references that are in the map. First posted on arXiv 22 May 2022.

Builds on

5 of 32 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Meta-Cognition. An Inverse-Inverse Reinforcement Learning Approach for Cognitive Radars3 May

Other references · 4

  1. 2020Opportunities and Challenges in Deep Learning Adversarial Robustness: A Surveycited by 72
  2. 2020Adversarial Radar Inference: Inverse Tracking, Identifying Cognition and Designing Smart Interferencecited by 10
  3. 2014Online Learning and Profit Maximization from Revealed Preferencescited by 6
  4. 2021How can a Cognitive Radar Mask its Cognition?cited by 4

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 1.

3 papers.

  1. 2022Inverse Extended Kalman Filter -- Part II: Highly Non-Linear and Uncertain Systemscited by 5
  2. 2022How can a Radar Mask its Cognition?cited by 1
  3. 2023Inverse Unscented Kalman Filtercited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress
  2. 2021Inverse Reinforcement Learning for Strategy Identification
  3. 2025Inverse Reinforcement Learning using Revealed Preferences and Passive Stochastic Optimization
  4. 2023Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective
  5. 2024Offline Inverse RL: New Solution Concepts and Provably Efficient Algorithms
  6. 2024Non-Adversarial Inverse Reinforcement Learning via Successor Feature Matching
  7. 2018A Framework and Method for Online Inverse Reinforcement Learning
  8. 2020oIRL: Robust Adversarial Inverse Reinforcement Learning with Temporally Extended Actions