# Human-in-the-Loop Testing of AI Agents for Air Traffic Control with a Regulated Assessment Framework

arXiv: https://arxiv.org/abs/2601.04288 · HTML: https://mlsanity.com/paper/2601.04288/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 24 references that are in the map.
- First posted on arXiv 7 Jan 2026.

## Builds on (most cited first, 6 of 6)

- [The CMA Evolution Strategy: A Tutorial](https://mlsanity.com/paper/1604.00772/) (2016)
- [A Survey of Human-in-the-loop for Machine Learning](https://mlsanity.com/paper/2108.00941/) (2021)
- [Autonomous Air Traffic Controller: A Deep Multi-Agent Reinforcement Learning Approach](https://mlsanity.com/paper/1905.01303/) (2019)
- [A Probabilistic Digital Twin of UK En Route Airspace for Training and Evaluating AI Agents for Air Traffic Control](https://mlsanity.com/paper/2601.03113/) (2026)
- [A framework for assuring the accuracy and fidelity of an AI-enabled Digital Twin of en route UK airspace](https://mlsanity.com/paper/2601.03120/) (2026)
- [Fast Surrogate Models for Adaptive Aircraft Trajectory Prediction in En route Airspace](https://mlsanity.com/paper/2601.03075/) (2026)

## Built on by (most cited first, 1 of 1)

- [From Automation to Collaboration: Human-in-the-Loop Methods for Safe and Trustworthy NLP](https://mlsanity.com/paper/2605.25226/) (2026)

## Abstract (arXiv)

We present a rigorous, human-in-the-loop evaluation framework for assessing the performance of AI agents on the task of Air Traffic Control, grounded in a regulator-certified simulator-based curriculum used for training and testing real-world trainee controllers. By leveraging legally regulated assessments and involving expert human instructors in the evaluation process, our framework enables a more authentic and domain-accurate measurement of AI performance. This work addresses a critical gap in the existing literature: the frequent misalignment between academic representations of Air Traffic Control and the complexities of the actual operational environment. It also lays the foundations for effective future human-machine teaming paradigms by aligning machine performance with human assessment targets.