Eliciting ESG Preferences for Reinforcement Learning-Based Portfolio Optimization
On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods
Abstract (arXiv)
Modern portfolio management increasingly demands a balance between traditional risk-adjusted returns and strict Environmental, Social, and Governance (ESG) mandates. Current Reinforcement Learning (RL) approaches typically optimize for a single ESG provider, neglecting the significant divergence in rating methodologies across the industry and the unintuitive nature of manually weighting conflicting objectives. This paper addresses these limitations by formulating ESG-aware portfolio optimization as a Multi-Objective Reinforcement Learning (MORL) problem that simultaneously incorporates ratings from three distinct ESG agencies. To bridge the gap between high-dimensional algorithmic trade-offs and human decision-making, we integrate a Preference Elicitation framework using Gaussian Processes. This system enables practitioners to infer their latent utility functions through intuitive pairwise comparisons of candidate portfolios based on their Sharpe ratios and aggregate ESG scores. We systematically evaluate our framework by employing Large Language Model (LLM) personas to simulate Portfolio Managers operating under varied regional contexts. Empirical results using historical market data reveal that regional backgrounds fundamentally shift the derived preference weights. For instance, European-based personas tend to prioritize ESG alignment over financial returns, while Texas-based personas favor risk-adjusted performance. This work offers a highly adaptable framework that successfully aligns multi-objective algorithmic trading with diverse, real-world human sustainability preferences.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 12 of its 28 references that are in the map. First posted on arXiv 2 Sep 2026.
Builds on
12 of 28 references.
References · 12
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
- 2014A Survey of Multi-Objective Sequential Decision-Makingcited by 229
- 2023Better Zero-Shot Reasoning with Role-Play Promptingcited by 220
- 2024Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalizationcited by 203
- 2018Market Making via Reinforcement Learningcited by 45
- 2023qEUBO: A Decision-Theoretic Acquisition Function for Preferential Bayesian Optimizationcited by 23
- 2020Option Hedging with Risk Averse Reinforcement Learningcited by 13
- 2024Deep Reinforcement Learning and Mean-Variance Strategies for Responsible Portfolio Optimizationcited by 4
- 2025Prompting for Policy: Forecasting Macroeconomic Scenarios with Synthetic LLM Personascited by 4
- 2023Deep Reinforcement Learning for ESG financial portfolio managementcited by 2
- 2026Deep Reinforcement Learning for Optimal Portfolio Allocation: A Comparative Study with Mean-Variance Optimizationcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2022Combining Reinforcement Learning and Inverse Reinforcement Learning for Asset Allocation Recommendations
- 2025Diffusion-Augmented Reinforcement Learning for Robust Portfolio Optimization under Stress Scenarios
- 2023PRUDEX-Compass: Towards Systematic Evaluation of Reinforcement Learning in Financial Markets
- 2025Learning to Manage Investment Portfolios beyond Simple Utility Functions
- 2025Reinforcement Learning for Portfolio Optimization with a Financial Goal and Defined Time Horizons
- 2026Hindsight Preference Optimization for Financial Time Series Advisory
- 2025Alpha-R1: Alpha Screening with LLM Reasoning via Reinforcement Learning
- 2026Regime-Aware Portfolio Management via Retrieval-Augmented LLM-Guided Expert Switching