mlsanityData to 7 Oct 2026

Reinforcement Learning with Trajectory Feedback

arXiv 2008.06036 ↗First posted 13 Aug 2020cs.LG stat.MLcited by 40

Not on the map. Why?

Abstract (arXiv)

The standard feedback model of reinforcement learning requires revealing the reward of every visited state-action pair. However, in practice, it is often the case that such frequent feedback is not available. In this work, we take a first step towards relaxing this assumption and require a weaker form of feedback, which we refer to as trajectory feedback. Instead of observing the reward obtained after every action, we assume we only receive a score that represents the quality of the whole trajectory observed by the agent, namely, the sum of all rewards obtained over this trajectory. We extend reinforcement learning algorithms to this setting, based on least-squares estimation of the unknown reward, for both the known and unknown transition model cases, and study the performance of these algorithms by analyzing their regret. For cases where the transition model is unknown, we offer a hybrid optimistic-Thompson Sampling approach that results in a tractable algorithm.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Aug 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 5; 2023: 8; 2024: 10; 2025: 8; 2026: 2.

40 papers.

Sort:

By area:

  1. 2023Nash Learning from Human Feedbackcited by 220
  2. 2021Dueling RL: Reinforcement Learning with Trajectory Preferencescited by 105
  3. 2022Human-in-the-loop: Provably Efficient Preference-based Reinforcement Learning with General Function Approximationcited by 80
  4. 2024Multi-turn Reinforcement Learning from Preference Human Feedbackcited by 65
  5. 2021Learning Long-Term Reward Redistribution via Randomized Return Decompositioncited by 33
  6. 2021On the Theory of Reinforcement Learning with Once-per-Episode Feedbackcited by 32
  7. 2021Minimax Regret for Stochastic Shortest Pathcited by 30
  8. 2022Challenging Common Assumptions in Convex Reinforcement Learningcited by 29
  9. 2021Provable RL with Exogenous Distractors via Multistep Inverse Dynamicscited by 28
  10. 2023Learning Energy Decompositions for Partial Inference of GFlowNetscited by 24
  11. 2024Latent Reward: LLM-Empowered Credit Assignment in Episodic Reinforcement Learningcited by 22
  12. 2022Provably Efficient Offline Reinforcement Learning with Trajectory-Wise Rewardcited by 13
28 more
  1. 2021Confidence-Budget Matching for Sequential Budgeted Learningcited by 11
  2. 2023Reinforcement Learning with History-Dependent Dynamic Contextscited by 11
  3. 2021Online Markov Decision Processes with Aggregate Bandit Feedbackcited by 10
  4. 2023Would I have gotten that reward? Long-term credit assignment by counterfactual contribution analysiscited by 10
  5. 2024Near-Optimal Regret in Linear MDPs with Aggregate Bandit Feedbackcited by 10
  6. 2023Consistent Aggregation of Objectives with Diverse Time Preferences Requires Non-Markovian Rewardscited by 8
  7. 2024Assessing the Zero-Shot Capabilities of LLMs for Action Evaluation in RLcited by 7
  8. 2025Outcome-Based Online Reinforcement Learning: Algorithms and Fundamental Limitscited by 7
  9. 2023Towards Fairness in Personalized Ads Using Impression Variance Aware Reinforcement Learningcited by 6
  10. 2024Test-Time Regret Minimization in Meta Reinforcement Learningcited by 5
  11. 2022Reinforcement Learning with a Terminatorcited by 4
  12. 2024Reinforcement Learning from Human Feedback without Reward Inference: Model-Free Algorithm and Instance-Dependent Analysiscited by 4
  13. 2025Reinforcement Learning with Segment Feedbackcited by 4
  14. 2025Zero-Shot LLMs in Human-in-the-Loop RL: Replacing Human Feedback for Reward Shapingcited by 3
  15. 2025Efficient Preference-Based Reinforcement Learning: Randomized Exploration Meets Experimental Designcited by 3
  16. 2023Iterative Reward Shaping using Human Feedback for Correcting Reward Misspecificationcited by 2
  17. 2023Episodic Return Decomposition by Difference of Implicitly Assigned Sub-Trajectory Rewardcited by 2
  18. 2024Beyond Simple Sum of Delayed Rewards: Non-Markovian Reward Modeling for Reinforcement Learningcited by 2
  19. 2025Adapting to Stochastic and Adversarial Losses in Episodic MDPs with Aggregate Bandit Feedbackcited by 2
  20. 2022A Unified Algorithm for Stochastic Path Problemscited by 1
  21. 2024ELO-Rated Sequence Rewards: Advancing Reinforcement Learning Modelscited by 1
  22. 2024Agent-Temporal Credit Assignment for Optimal Policy Preservation in Sparse Multi-Agent Reinforcement Learningcited by 1
  23. 2024Preference-Guided Reinforcement Learning for Efficient Explorationcited by 0
  24. 2025A representational framework for learning and encoding structurally enriched trajectories in complex agent environmentscited by 0
  25. 2025Reinforcement Learning from Multi-level and Episodic Human Feedbackcited by 0
  26. 2025Adversarial Reinforcement Learning Framework for ESP Cheater Simulationcited by 0
  27. 2026Learning Markov Decision Processes under Fully Bandit Feedbackcited by 0
  28. 2026Learning Kernel-Based MDPs from Episodic Preferential Feedbackcited by 0