# GIFT: LLM-Guided State-Reward Interface for Financial Reinforcement Learning

arXiv: https://arxiv.org/abs/2606.08450 · HTML: https://mlsanity.com/paper/2606.08450/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 42 references that are in the map.
- First posted on arXiv 7 Jun 2026.

## Builds on (most cited first, 16 of 16)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [BloombergGPT: A Large Language Model for Finance](https://mlsanity.com/paper/2303.17564/) (2023)
- [A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem](https://mlsanity.com/paper/1706.10059/) (2017)
- [Recent Advances in Reinforcement Learning in Finance](https://mlsanity.com/paper/2112.04553/) (2021)
- [Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraft](https://mlsanity.com/paper/2312.09238/) (2023)
- [LLM-Empowered State Representation for Reinforcement Learning](https://mlsanity.com/paper/2407.13237/) (2024)
- [FLAG-Trader: Fusion LLM-Agent with Gradient-based Reinforcement Learning for Financial Trading](https://mlsanity.com/paper/2502.11433/) (2025)
- [DoubleAdapt: A Meta-learning Approach to Incremental Learning for Stock Trend Forecasting](https://mlsanity.com/paper/2306.09862/) (2023)
- [MAPS: Multi-agent Reinforcement Learning-based Portfolio Management System](https://mlsanity.com/paper/2007.05402/) (2020)
- [The Evolution of Reinforcement Learning in Quantitative Finance: A Survey](https://mlsanity.com/paper/2408.10932/) (2024)
- [Language Model Guided Reinforcement Learning in Quantitative Trading](https://mlsanity.com/paper/2508.02366/) (2025)
- [FinLLM-B: When Large Language Models Meet Financial Breakout Trading](https://mlsanity.com/paper/2402.07536/) (2024)
- [Robot See, Robot Do: Imitation Reward for Noisy Financial Environments](https://mlsanity.com/paper/2411.08637/) (2024)
- [FinRLlama: A Solution to LLM-Engineered Signals Challenge at FinRL Contest 2024](https://mlsanity.com/paper/2502.01992/) (2025)
- [Regret-Optimized Portfolio Enhancement through Deep Reinforcement Learning and Future Looking Rewards](https://mlsanity.com/paper/2502.02619/) (2025)

## Built on by (most cited first, 1 of 1)

- [Risk-Aware Reranking for Agentic Tool Retrieval](https://mlsanity.com/paper/2608.22751/) (2026)

## Abstract (arXiv)

Financial portfolio trading is naturally formulated as a reinforcement learning problem, where an agent sequentially rebalances assets under changing market conditions to balance return, risk, and transaction costs. Yet in non-stationary markets, raw OHLCV states and short-horizon return rewards often provide an under-specified learning interface, motivating large language models as a way to inject financial knowledge into state and reward design while constraining open-ended generation. To this end, we propose GIFT, an LLM-guided framework for state-reward interface design in PPO-based financial reinforcement learning. Rather than using the LLM to make trading decisions, GIFT uses Factor-guided State Enhancement to generate state features from financial-factor primitives, Risk-rule-guided Reward Shaping to generate auxiliary rewards from portfolio-risk rules, and Diagnostic-guided Refinement to revise candidate interfaces using PPO rollout diagnostics. After refinement, GIFT fixes the selected state-reward interface before evaluation, with no further LLM queries or interface updates at test time. Comprehensive rolling-window experiments across diverse market regimes and portfolio scenarios demonstrate that GIFT improves learning-signal quality and out-of-sample risk-adjusted portfolio performance over baselines. Code and data are available at: https://github.com/KAG778/GIFT .