mlsanityData to 7 Oct 2026

Trajectory Data Suffices for Statistically Efficient Policy Evaluation in Fixed-Horizon Offline RL with Linear qπ-Realizability and Concentrability

arXiv 2510.03494 ↗First posted 3 Oct 2025cs.LG stat.MLcited by 0

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

We study finite-horizon offline reinforcement learning (RL) with function approximation for both policy evaluation and policy optimization. Prior work established that statistically efficient learning is impossible for either of these problems when the only assumptions are that the data has good coverage (concentrability) and the state-action value function of every policy is linearly realizable (qπ-realizability) (Foster et al., 2021). Recently, Tkachuk et al. (2024) gave a statistically efficient learner for policy optimization, if in addition the data is assumed to be given as trajectories. In this work we present a statistically efficient learner for policy evaluation under the same assumptions. Further, we show that the sample complexity of the learner used by Tkachuk et al. (2024) for policy optimization can be improved by a tighter analysis.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 11 of its 23 references that are in the map. First posted on arXiv 3 Oct 2025.

Builds on

11 of 23 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Trajectory Data Suffices for Statistically Efficient Learning in Offline RL with Linear qπ-Realizability and Concentrability27 May
  2. 2024Offline Reinforcement Learning: Role of State Aggregation and Trajectory Data25 Mar
  3. 2022Offline Reinforcement Learning with Realizability and Single-policy Concentrability9 Feb
  4. 2021Offline Reinforcement Learning: Fundamental Barriers for Value Function Approximation21 Nov

Other references · 7

  1. 2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 514
  2. 2019Information-Theoretic Considerations in Batch Reinforcement Learningcited by 285
  3. 2019Batch Policy Learning under Constraintscited by 242
  4. 2019Provably Efficient Exploration in Policy Optimizationcited by 203
  5. 2020Learning Near Optimal Policies with Low Inherent Bellman Errorcited by 180
  6. 2024Sample and Oracle Efficient Reinforcement Learning for MDPs with Linearly-Realizable Value Functionscited by 3
  7. 2023Online RL in Linearly qπ-Realizable MDPs Is as Easy as in Linear MDPs If You Learn What to Ignorecited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2020What are the Statistical Limits of Offline RL with Linear Function Approximation?
  2. 2021Infinite-Horizon Offline Reinforcement Learning with Linear Function Approximation: Curse of Dimensionality and Algorithm
  3. 2025On the Statistical Complexity for Offline and Low-Adaptive Reinforcement Learning with Structures
  4. 2023Offline Reinforcement Learning with Additional Covering Distributions
  5. 2026On the Complexity of Offline Reinforcement Learning with Q^star-Approximation and Partial Coverage
  6. 2023Offline Policy Evaluation for Reinforcement Learning with Adaptively Collected Data
  7. 2022Settling the Sample Complexity of Model-Based Offline Reinforcement Learning
  8. 2024On Sample-Efficient Offline Reinforcement Learning: Data Diversity, Posterior Sampling, and Beyond