# An LLM-Based Digital Twin for Optimizing Human-in-the Loop Systems

arXiv: https://arxiv.org/abs/2403.16809 · HTML: https://mlsanity.com/paper/2403.16809/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 17 references that are in the map.
- First posted on arXiv 25 Mar 2024.

## Builds on (most cited first, 6 of 6)

- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies](https://mlsanity.com/paper/2208.10264/) (2022)
- [Exploring Large Language Models for Communication Games: An Empirical Study on Werewolf](https://mlsanity.com/paper/2309.04658/) (2023)
- [LLM-Based Agent Society Investigation: Collaboration and Confrontation in Avalon Gameplay](https://mlsanity.com/paper/2310.14985/) (2023)
- [Towards autonomous system: flexible modular production system enhanced with large language model agents](https://mlsanity.com/paper/2304.14721/) (2023)

## Built on by (most cited first, 13 of 13)

- [LLM-Powered Decentralized Generative Agents with Adaptive Hierarchical Knowledge Graph for Cooperative Planning](https://mlsanity.com/paper/2502.05453/) (2025)
- [Can LLMs Correct Themselves? A Benchmark of Self-Correction in LLMs](https://mlsanity.com/paper/2510.16062/) (2025)
- [Large Language Models for Explainable Decisions in Dynamic Digital Twins](https://mlsanity.com/paper/2405.14411/) (2024)
- [How Social is It? A Benchmark for LLMs' Capabilities in Multi-user Multi-turn Social Agent Tasks](https://mlsanity.com/paper/2505.04628/) (2025)
- [Digital Twin AI: Opportunities and Challenges from Large Language Models to World Models](https://mlsanity.com/paper/2601.01321/) (2026)
- [Towards Edge General Intelligence via Large Language Models: Opportunities and Challenges](https://mlsanity.com/paper/2410.18125/) (2024)
- [Exploring Big Five Personality and AI Capability Effects in LLM-Simulated Negotiation Dialogues](https://mlsanity.com/paper/2506.15928/) (2025)
- [Tapping in a Remote Vehicle's onboard LLM to Complement the Ego Vehicle's Field-of-View](https://mlsanity.com/paper/2408.10794/) (2024)
- [Foundation-Model-Based Agents in Industrial Automation: Purposes, Capabilities, and Open Challenges](https://mlsanity.com/paper/2605.02592/) (2026)
- [SCoTT: Strategic Chain-of-Thought Tasking for Wireless-Aware Robot Navigation in Digital Twins](https://mlsanity.com/paper/2411.18212/) (2024)
- [Agentic Driving Coach: Robustness and Determinism of Agentic AI-Powered Human-in-the-Loop Cyber-Physical Systems](https://mlsanity.com/paper/2604.11705/) (2026)
- [AgenticTwin: An Agentic LLM Framework Integrated with Digital Twin for Anomaly Detection](https://mlsanity.com/paper/2608.11679/) (2026)
- [Pincer: Resource Authorization for Agents using a Digital Twin](https://mlsanity.com/paper/2610.02569/) (2026)

## Abstract (arXiv)

The increasing prevalence of Cyber-Physical Systems and the Internet of Things (CPS-IoT) applications and Foundation Models are enabling new applications that leverage real-time control of the environment. For example, real-time control of Heating, Ventilation and Air-Conditioning (HVAC) systems can reduce its usage when not needed for the comfort of human occupants, hence reducing energy consumption. Collecting real-time feedback on human preferences in such human-in-the-loop (HITL) systems, however, is difficult in practice. We propose the use of large language models (LLMs) to deal with the challenges of dynamic environments and difficult-to-obtain data in CPS optimization. In this paper, we present a case study that employs LLM agents to mimic the behaviors and thermal preferences of various population groups (e.g. young families, the elderly) in a shopping mall. The aggregated thermal preferences are integrated into an agent-in-the-loop based reinforcement learning algorithm AitL-RL, which employs the LLM as a dynamic simulation of the physical environment to learn how to balance between energy savings and occupant comfort. Our results show that LLMs are capable of simulating complex population movements within large open spaces. Besides, AitL-RL demonstrates superior performance compared to the popular existing policy of set point control, suggesting that adaptive and personalized decision-making is critical for efficient optimization in CPS-IoT applications. Through this case study, we demonstrate the potential of integrating advanced Foundation Models like LLMs into CPS-IoT to enhance system adaptability and efficiency. The project's code can be found on our GitHub repository.