mlsanityData to 7 Oct 2026

Automated Reward Design for Gran Turismo

arXiv 2511.02094 ↗First posted 3 Nov 2025cs.AIcited by 2

On the map: LLM alignment / RLHF / preference optimisation → Reward modeling and RLHF

Abstract (arXiv)

When designing reinforcement learning (RL) agents, a designer communicates the desired agent behavior through the definition of reward functions - numerical feedback given to the agent as reward or punishment for its actions. However, mapping desired behaviors to reward functions can be a difficult process, especially in complex environments such as autonomous racing. In this paper, we demonstrate how current foundation models can effectively search over a space of reward functions to produce desirable RL agents for the Gran Turismo 7 racing game, given only text-based instructions. Through a combination of LLM-based reward generation, VLM preference-based evaluation, and human feedback we demonstrate how our system can be used to produce racing agents competitive with GT Sophy, a champion-level RL racing agent, as well as generate novel behaviors, paving the way for practical automated reward design in real world applications.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 33 of its 48 references that are in the map. First posted on arXiv 3 Nov 2025.

Builds on

33 of 48 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024A Large Language Model-Driven Reward Design Framework via Dynamic Feedback for Reinforcement Learning18 Oct
  2. 2024REvolve: Reward Evolution with Large Language Models using Human Feedback3 Jun
  3. 2023Eureka: Human-Level Reward Design via Coding Large Language Models19 Oct
  4. 2023Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learning19 Oct
  5. 2023Text2Reward: Reward Shaping with Language Models for Reinforcement Learning20 Sep

Other references · 28

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  5. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
  6. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  7. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 342
  8. 2023Language to Rewards for Robotic Skill Synthesiscited by 307
  9. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 300
  10. 2023Reward Design with Language Modelscited by 257
  11. 2022The Primacy Bias in Deep Reinforcement Learningcited by 242
  12. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 153
16 more
  1. 2023RoboCLIP: One Demonstration is Enough to Learn Robot Policiescited by 128
  2. 2021Reward (Mis)design for Autonomous Drivingcited by 102
  3. 2023Motif: Intrinsic Motivation from Artificial Intelligence Feedbackcited by 99
  4. 2023Video Prediction Models as Rewards for Reinforcement Learningcited by 76
  5. 2020Super-Human Performance in Gran Turismo Sport Using Deep Reinforcement Learningcited by 60
  6. 2023Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraftcited by 60
  7. 2024VLM-RL: A Unified Vision Language Models and Reinforcement Learning Framework for Safe Autonomous Drivingcited by 49
  8. 2021Autonomous Overtaking in Gran Turismo Sport Using Curriculum Reinforcement Learningcited by 41
  9. 2024MaestroMotif: Skill Design from Artificial Intelligence Feedbackcited by 24
  10. 2022EAGER: Asking and Answering Questions for Automatic Reward Shaping in Language-guided RLcited by 23
  11. 2021Formula RL: Deep Reinforcement Learning for Autonomous Racing using Telemetry Datacited by 10
  12. 2023Replay across Experiments: A Natural Extension of Off-Policy RLcited by 10
  13. 2025Towards Improving Reward Design in RL: A Reward Alignment Metric for RL Practitionerscited by 10
  14. 2024Adapt2Reward: Adapting Video-Language Models to Generalizable Robotic Rewards via Failure Promptscited by 9
  15. 2024ORSO: Accelerating Reward Design via Online Reward Selection and Policy Optimizationcited by 6
  16. 2025A Champion-level Vision-based Reinforcement Learning Agent for Competitive Racing in Gran Turismo 7cited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Differentiable Evolutionary Reinforcement Learningcited by 1
  2. 2026RDA: Reward Design Agent for Reinforcement Learningcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Self-correcting Reward Shaping via Language Models for Reinforcement Learning Agents in Games
  2. 2024Human-centric Reward Optimization for Reinforcement Learning-based Automated Driving using Large Language Models
  3. 2025RLAF: Reinforcement Learning from Automaton Feedback
  4. 2025Boosting Universal LLM Reward Design through Heuristic Reward Observation Space Evolution
  5. 2024Language Models as Efficient Reward Function Searchers for Custom-Environment Multi-Objective Reinforcement
  6. 2024Generating and Evolving Reward Functions for Highway Driving with Large Language Models
  7. 2024RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedback
  8. 2021Direct Behavior Specification via Constrained Reinforcement Learning