mlsanityData to 14 Oct 2026

Efficiently Generating Expressive Quadruped Behaviors via Language-Guided Preference Learning

arXiv 2502.03717 ↗First posted 6 Feb 2025cs.RO cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Expressive robotic behavior is essential for the widespread acceptance of robots in social environments. Recent advancements in learned legged locomotion controllers have enabled more dynamic and versatile robot behaviors. However, determining the optimal behavior for interactions with different users across varied scenarios remains a challenge. Current methods either rely on natural language input, which is efficient but low-resolution, or learn from human preferences, which, although high-resolution, is sample inefficient. This paper introduces a novel approach that leverages priors generated by pre-trained LLMs alongside the precision of preference learning. Our method, termed Language-Guided Preference Learning (LGPL), uses LLMs to generate initial behavior samples, which are then refined through preference-based feedback to learn behaviors that closely align with human expectations. Our core insight is that LLMs can guide the sampling process for preference learning, leading to a substantial improvement in sample efficiency. We demonstrate that LGPL can quickly learn accurate and expressive behaviors with as few as four queries, outperforming both purely language-parameterized models and traditional preference learning approaches. Website with videos: https://lgpl-gaits.github.io/

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 35 of its 40 references that are in the map. First posted on arXiv 6 Feb 2025.

Builds on

35 of 40 references.

References · 35

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2017Deep reinforcement learning from human preferencescited by 3,912
  5. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  6. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  7. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
  8. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  9. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 759
  10. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  11. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 571
  12. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
23 more
  1. 2023Language to Rewards for Robotic Skill Synthesiscited by 312
  2. 2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-trainingcited by 302
  3. 2023Reward Design with Language Modelscited by 261
  4. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 260
  5. 2017Inverse Reward Designcited by 249
  6. 2017Interactive Learning from Policy-Dependent Human Feedbackcited by 158
  7. 2022Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behaviorcited by 138
  8. 2022SURF: Semi-supervised Reward Learning with Data Augmentation for Feedback-efficient Preference-based Reinforcement Learningcited by 97
  9. 2018Batch Active Preference-Based Learning of Reward Functionscited by 87
  10. 2022Few-Shot Preference Learning for Human-in-the-Loop RLcited by 87
  11. 2021Minimizing Energy Consumption Leads to the Emergence of Gaits in Legged Robotscited by 76
  12. 2020Safe Imitation Learning via Fast Bayesian Reward Inference from Preferencescited by 73
  13. 2023Language Instructed Reinforcement Learning for Human-AI Coordinationcited by 63
  14. 2021Fast and Efficient Locomotion via Learned Gait Transitionscited by 54
  15. 2021Learning Multimodal Rewards from Rankingscited by 41
  16. 2023Barkour: Benchmarking Animal-level Agility with Quadruped Robotscited by 41
  17. 2023SayTap: Language to Quadrupedal Locomotioncited by 41
  18. 2024Learning to Learn Faster from Human Feedback with Language Model Predictive Controlcited by 26
  19. 2024Generative Expressive Robot Behaviors using Large Language Modelscited by 22
  20. 2021Learning Human Objectives from Sequences of Physical Correctionscited by 21
  21. 2024Preference-Conditioned Language-Guided Abstractioncited by 12
  22. 2024MAPLE: A Framework for Active Preference Learning Guided by Large Language Modelscited by 6
  23. 2024Text2Interaction: Establishing Safe and Preferable Human-Robot Interactioncited by 4

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Learning Acrobatic Flight from Preferencescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025LAPP: Large Language Model Feedback for Preference-Driven Reinforcement Learning
  2. 2024PrefMMT: Modeling Human Preferences in Preference-based Reinforcement Learning with Multimodal Transformers
  3. 2026MAPL: Multi-Objective Preference Learning for Robot Locomotion ↗
  4. 2024PREDILECT: Preferences Delineated with Zero-Shot Language-based Reasoning in Reinforcement Learning
  5. 2024PrefCLM: Enhancing Preference-based Reinforcement Learning with Crowdsourced Large Language Models ↗
  6. 2024Sample-Efficient Preference-based Reinforcement Learning with Dynamics Aware Rewards
  7. 2025PRIMT: Preference-based Reinforcement Learning with Multimodal Feedback and Trajectory Synthesis from Foundation Models
  8. 2025VARP: Reinforcement Learning from Vision-Language Model Feedback with Agent Regularized Preferences