mlsanityData to 14 Oct 2026

Learning Trajectory Preferences for Manipulators via Iterative Improvement

arXiv 1306.6294 ↗First posted 26 Jun 2013cs.RO cs.AI cs.HCcited by 31

Not on the map. Why?

Abstract (arXiv)

We consider the problem of learning good trajectories for manipulation tasks. This is challenging because the criterion defining a good trajectory varies with users, tasks and environments. In this paper, we propose a co-active online learning framework for teaching robots the preferences of its users for object manipulation tasks. The key novelty of our approach lies in the type of feedback expected from the user: the human user does not need to demonstrate optimal trajectories as training data, but merely needs to iteratively provide trajectories that slightly improve over the trajectory currently proposed by the system. We argue that this co-active preference feedback can be more easily elicited from the user than demonstrations of optimal trajectories, which are often challenging and non-intuitive to provide on high degrees of freedom manipulators. Nevertheless, theoretical regret bounds of our algorithm match the asymptotic rates of optimal trajectory algorithms. We demonstrate the generalizability of our algorithm on a variety of grocery checkout tasks, for whom, the preferences were not only influenced by the object being manipulated but also by the surrounding environment.footnoteFor more details and a demonstration video, visit: urlhttp://pr.cs.cornell.edu/coactive

Key facts. Cited by 31 papers in the map (data through 14 Oct 2026). First posted on arXiv 26 Jun 2013.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 1; 2023: 7; 2024: 10; 2025: 6; 2026: 6.

31 papers.

Sort:

By area:

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  2. 2024KTO: Model Alignment as Prospect Theoretic Optimizationcited by 264
  3. 2023Principled Reinforcement Learning with Human Feedback from Pairwise or K-wise Comparisonscited by 247
  4. 2021Dueling RL: Reinforcement Learning with Trajectory Preferencescited by 106
  5. 2023Is RLHF More Difficult than Standard RL?cited by 96
  6. 2022Human-in-the-loop: Provably Efficient Preference-based Reinforcement Learning with General Function Approximationcited by 81
  7. 2023Reinforcement Learning with Human Feedback: Learning Dynamic Choices via Pessimismcited by 69
  8. 2024TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correctioncited by 59
  9. 2023Pairwise Proximal Policy Optimization: Harnessing Relative Feedback for LLM Alignmentcited by 57
  10. 2024Iterative Data Smoothing: Mitigating Reward Overfitting and Overoptimization in RLHFcited by 57
  11. 2024Enhancing Zero-shot Text-to-Speech Synthesis with Human Feedbackcited by 20
  12. 2024Preference Poisoning Attacks on Reward Model Learningcited by 14
19 more
  1. 2024DrS: Learning Reusable Dense Rewards for Multi-Stage Taskscited by 10
  2. 2024Near-Optimal Regret in Linear MDPs with Aggregate Bandit Feedbackcited by 10
  3. 2024RA-PbRL: Provably Efficient Risk-Aware Preference-Based Reinforcement Learningcited by 9
  4. 2025Enhancing Decision-Making of Large Language Models via Actor-Criticcited by 7
  5. 2026GOPO: Policy Optimization using Ranked Rewardscited by 5
  6. 2024On The Global Convergence Of Online RLHF With Neural Parametrizationcited by 3
  7. 2025Efficient Preference-Based Reinforcement Learning: Randomized Exploration Meets Experimental Designcited by 3
  8. 2026Safe and Interpretable Multimodal Path Planning for Multi-Agent Cooperationcited by 3
  9. 2023COPR: Continual Learning Human Preference through Optimal Policy Regularizationcited by 2
  10. 2025Preference-based Reinforcement Learning beyond Pairwise Comparisons: Benefits of Multiple Optionscited by 2
  11. 2024MEReQ: Max-Ent Residual-Q Inverse RL for Sample-Efficient Alignment from Interventioncited by 1
  12. 2025Enhancing Speech Large Language Models through Reinforced Behavior Alignmentcited by 1
  13. 2026Enhancing Goal Inference via Correction Timingcited by 1
  14. 2026Regret Bounds for Reinforcement Learning from Multi-Source Imperfect Preferencescited by 1
  15. 2023Sample Complexity of Preference-Based Nonparametric Off-Policy Evaluation with Deep Networkscited by 0
  16. 2025Reinforcement Learning from Multi-level and Episodic Human Feedbackcited by 0
  17. 2025Thompson Sampling in Online RLHF with General Function Approximationcited by 0
  18. 2026Riemannian Dueling Optimizationcited by 0
  19. 2026Preference-Guided Adaptation for Open-Vocabulary Semantic Segmentation via Prompt Disagreementcited by 0