# FaiR-IoT: Fairness-aware Human-in-the-Loop Reinforcement Learning for Harnessing Human Variability in Personalized IoT

arXiv: https://arxiv.org/abs/2103.16033 · HTML: https://mlsanity.com/paper/2103.16033/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 63 references that are in the map.
- First posted on arXiv 30 Mar 2021.

## Builds on (most cited first, 8 of 8)

- [Data-Efficient Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1805.08296/) (2018)
- [Cooperative Inverse Reinforcement Learning](https://mlsanity.com/paper/1606.03137/) (2016)
- [Fairness in Learning: Classic and Contextual Bandits](https://mlsanity.com/paper/1605.07139/) (2016)
- [Inequity aversion improves cooperation in intertemporal social dilemmas](https://mlsanity.com/paper/1803.08884/) (2018)
- [Fairness in Reinforcement Learning](https://mlsanity.com/paper/1611.03071/) (2016)
- [Learning Fairness in Multi-Agent Systems](https://mlsanity.com/paper/1910.14472/) (2019)
- [Hyp-RL : Hyperparameter Optimization by Reinforcement Learning](https://mlsanity.com/paper/1906.11527/) (2019)
- [Reinforcement Learning for Learning Rate Control](https://mlsanity.com/paper/1705.11159/) (2017)

## Built on by (most cited first, 15 of 15)

- [Survey on Fair Reinforcement Learning: Theory and Practice](https://mlsanity.com/paper/2205.10032/) (2022)
- [The Pursuit of Fairness in Artificial Intelligence Models: A Survey](https://mlsanity.com/paper/2403.17333/) (2024)
- [Fairness in Reinforcement Learning: A Survey](https://mlsanity.com/paper/2405.06909/) (2024)
- [adaPARL: Adaptive Privacy-Aware Reinforcement Learning for Sequential-Decision Making Human-in-the-Loop Systems](https://mlsanity.com/paper/2303.04257/) (2023)
- [ERUDITE: Human-in-the-Loop IoT for an Adaptive Personalized Learning System](https://mlsanity.com/paper/2303.04292/) (2023)
- [AutoFR: Automated Filter Rule Generation for Adblocking](https://mlsanity.com/paper/2202.12872/) (2022)
- [Enhancing the Hierarchical Environment Design via Generative Trajectory Modeling](https://mlsanity.com/paper/2310.00301/) (2023)
- [FinP: Fairness-in-Privacy in Federated Learning by Addressing Disparities in Privacy Risk](https://mlsanity.com/paper/2502.17748/) (2025)
- [Improving the Efficiency of Human-in-the-Loop Systems: Adding Artificial to Human Experts](https://mlsanity.com/paper/2307.03003/) (2023)
- [FAIRO: Fairness-aware Adaptation in Sequential-Decision Making for Human-in-the-Loop Systems](https://mlsanity.com/paper/2307.05857/) (2023)
- [PEARL: Structural Privacy-Utility Control in Human-Centric CPS via Personalized Early-Exit Deep Reinforcement Learning](https://mlsanity.com/paper/2403.05864/) (2024)
- [Unbiasing on the Fly: Explanation-Guided Human Oversight of Machine Learning System Decisions](https://mlsanity.com/paper/2406.17906/) (2024)
- [FinA: Fairness of Adverse Effects in Decision-Making of Human-Cyber-Physical-System](https://mlsanity.com/paper/2311.03468/) (2023)
- [Off-Policy Selection for Initiating Human-Centric Experimental Design](https://mlsanity.com/paper/2410.20017/) (2024)
- [Fair Resource Allocation in Weakly Coupled Markov Decision Processes](https://mlsanity.com/paper/2411.09804/) (2024)

## Abstract (arXiv)

Thanks to the rapid growth in wearable technologies, monitoring complex human context becomes feasible, paving the way to develop human-in-the-loop IoT systems that naturally evolve to adapt to the human and environment state autonomously. Nevertheless, a central challenge in designing such personalized IoT applications arises from human variability. Such variability stems from the fact that different humans exhibit different behaviors when interacting with IoT applications (intra-human variability), the same human may change the behavior over time when interacting with the same IoT application (inter-human variability), and human behavior may be affected by the behaviors of other people in the same environment (multi-human variability). To that end, we propose FaiR-IoT, a general reinforcement learning-based framework for adaptive and fairness-aware human-in-the-loop IoT applications. In FaiR-IoT, three levels of reinforcement learning agents interact to continuously learn human preferences and maximize the system's performance and fairness while taking into account the intra-, inter-, and multi-human variability. We validate the proposed framework on two applications, namely (i) Human-in-the-Loop Automotive Advanced Driver Assistance Systems and (ii) Human-in-the-Loop Smart House. Results obtained on these two applications validate the generality of FaiR-IoT and its ability to provide a personalized experience while enhancing the system's performance by 40%-60% compared to non-personalized systems and enhancing the fairness of the multi-human systems by 1.5 orders of magnitude.