# Online Pareto-Optimal Decision-Making for Complex Tasks using Active Inference

arXiv: https://arxiv.org/abs/2406.11984 · HTML: https://mlsanity.com/paper/2406.11984/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 67 references that are in the map.
- First posted on arXiv 17 Jun 2024.

## Builds on (most cited first, 19 of 19)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress](https://mlsanity.com/paper/1806.06877/) (2018)
- [A Practical Guide to Multi-Objective Reinforcement Learning and Planning](https://mlsanity.com/paper/2103.09568/) (2021)
- [A Tutorial on Kernel Density Estimation and Recent Advances](https://mlsanity.com/paper/1704.03924/) (2017)
- [Dynamic Weights in Multi-Objective Deep Reinforcement Learning](https://mlsanity.com/paper/1809.07803/) (2018)
- [Deep active inference agents using Monte-Carlo methods](https://mlsanity.com/paper/2006.04176/) (2020)
- [Active Inference in Robotics and Artificial Agents: Survey and Challenges](https://mlsanity.com/paper/2112.01871/) (2021)
- [Scaling active inference](https://mlsanity.com/paper/1911.10601/) (2019)
- [Multi-objective Bandits: Optimizing the Generalized Gini Index](https://mlsanity.com/paper/1706.04933/) (2017)
- [Designing Ecosystems of Intelligence from First Principles](https://mlsanity.com/paper/2212.01354/) (2022)
- [Multi-objective Contextual Bandit Problem with Similarity Information](https://mlsanity.com/paper/1803.04015/) (2018)
- [Multi-Objective Generalized Linear Bandits](https://mlsanity.com/paper/1905.12879/) (2019)
- [Optimal task and motion planning and execution for human-robot multi-agent systems in dynamic environments](https://mlsanity.com/paper/2303.14874/) (2023)
- [Modeling Human Decision-making in Generalized Gaussian Multi-armed Bandits](https://mlsanity.com/paper/1307.6134/) (2013)
- [Active Inference for Autonomous Decision-Making with Contextual Multi-Armed Bandits](https://mlsanity.com/paper/2209.09185/) (2022)
- [Observation-Augmented Contextual Multi-Armed Bandits for Robotic Search and Exploration](https://mlsanity.com/paper/2312.12583/) (2023)
- [Optimal Cost-Preference Trade-off Planning with Multiple Temporal Tasks](https://mlsanity.com/paper/2306.13222/) (2023)
- [Secure-by-Construction Optimal Path Planning for Linear Temporal Logic Tasks](https://mlsanity.com/paper/2010.14160/) (2020)

## Abstract (arXiv)

When a robot autonomously performs a complex task, it frequently must balance competing objectives while maintaining safety. This becomes more difficult in uncertain environments with stochastic outcomes. Enhancing transparency in the robot's behavior and aligning with user preferences are also crucial. This paper introduces a novel framework for multi-objective reinforcement learning that ensures safe task execution, optimizes trade-offs between objectives, and adheres to user preferences. The framework has two main layers: a multi-objective task planner and a high-level selector. The planning layer generates a set of optimal trade-off plans that guarantee satisfaction of a temporal logic task. The selector uses active inference to decide which generated plan best complies with user preferences and aids learning. Operating iteratively, the framework updates a parameterized learning model based on collected data. Case studies and benchmarks on both manipulation and mobile robots show that our framework outperforms other methods and (i) learns multiple optimal trade-offs, (ii) adheres to a user preference, and (iii) allows the user to adjust the balance between (i) and (ii).