# Objective-Behavior Alignment: Diagnostics for MORL Policy Selection

arXiv: https://arxiv.org/abs/2606.21321 · HTML: https://mlsanity.com/paper/2606.21321/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 38 references that are in the map.
- First posted on arXiv 19 Jun 2026.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://mlsanity.com/paper/2105.04906/) (2021)
- [Is Conditional Generative Modeling all you need for Decision-Making?](https://mlsanity.com/paper/2211.15657/) (2022)
- [Scalable agent alignment via reward modeling: a research direction](https://mlsanity.com/paper/1811.07871/) (2018)
- [A Practical Guide to Multi-Objective Reinforcement Learning and Planning](https://mlsanity.com/paper/2103.09568/) (2021)
- [Inverse Reward Design](https://mlsanity.com/paper/1711.02827/) (2017)
- [A Survey of Multi-Objective Sequential Decision-Making](https://mlsanity.com/paper/1402.0590/) (2014)
- [Reward (Mis)design for Autonomous Driving](https://mlsanity.com/paper/2104.13906/) (2021)
- [Scalar reward is not enough: A response to Silver, Singh, Precup and Sutton (2021)](https://mlsanity.com/paper/2112.15422/) (2021)
- [Contrastive Trajectory Similarity Learning with Dual-Feature Attention](https://mlsanity.com/paper/2210.05155/) (2022)
- [Multi-Objective Reinforcement Learning Based on Decomposition: A Taxonomy and Framework](https://mlsanity.com/paper/2311.12495/) (2023)
- [UniMASK: Unified Inference in Sequential Decision Problems](https://mlsanity.com/paper/2211.10869/) (2022)
- [What Lies beyond the Pareto Front? A Survey on Decision-Support Methods for Multi-Objective Optimization](https://mlsanity.com/paper/2311.11288/) (2023)
- [On Learning Informative Trajectory Embeddings for Imitation, Classification and Regression](https://mlsanity.com/paper/2501.09327/) (2025)
- [Contrast & Compress: Learning Lightweight Embeddings for Short Trajectories](https://mlsanity.com/paper/2506.02571/) (2025)
- [Navigating Trade-offs: Policy Summarization for Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2411.04784/) (2024)
- [CoMI-IRL: Contrastive Multi-Intention Inverse Reinforcement Learning](https://mlsanity.com/paper/2602.07496/) (2026)

## Abstract (arXiv)

Real-world decision-making often requires optimizing multiple competing objectives simultaneously. In reinforcement learning (RL), this is typically addressed by combining reward signals into a single scalar objective via a scalarization function, which can be fragile: small changes in the weights can induce drastically different policies. Multi-objective reinforcement learning (MORL) instead produces sets of policies that explicitly represent trade-offs between objectives. However, these policies are typically presented to the decision maker only through their value vectors, which can obscure substantial behavioral variation: policies that induce distinct trajectories may appear indistinguishable when evaluated solely by expected returns. We propose an exploratory diagnostic workflow that automatically highlights behavioral variation along the Pareto front that objective values alone do not reveal, providing both quantitative and visual tools to support policy inspection. We validate our approach on simple grid examples and scale it to continuous control benchmarks, demonstrating that it remains effective as problem complexity increases.