# Knowledge Distillation from Large Language Models for Household Energy Modeling

arXiv: https://arxiv.org/abs/2502.03034 · HTML: https://mlsanity.com/paper/2502.03034/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge distillation for LLMs, one of 12 directions in Knowledge distillation.
- First posted on arXiv 5 Feb 2025.

## Placed in

- Knowledge distillation / [Knowledge distillation for LLMs](https://mlsanity.com/distill/knowledge-distillation-for-llms/)

## Built on by (most cited first, 4 of 4)

- [A Survey on the Green Development of Large Models: From Resource-Efficient Architectures to Hardware-Software Co-Design](https://mlsanity.com/paper/2607.09084/) (2026)
- [Negotiating Comfort: Simulating Personality-Driven LLM Agents in Shared Residential Social Networks](https://mlsanity.com/paper/2507.09657/) (2025)
- [WattCouncil: Context-Aware Household Energy Scenario Generation With Governed LLMs](https://mlsanity.com/paper/2607.10720/) (2026)
- [LLMs and Agentic AI Systems for Smart Grids: A Tutorial on Architectures and Applications](https://mlsanity.com/paper/2607.18147/) (2026)

## Abstract (arXiv)

Machine learning (ML) is increasingly vital for smart-grid research, yet restricted access to realistic, diverse data - often due to privacy concerns - slows progress and fuels doubts within the energy sector about adopting ML-based strategies. We propose integrating Large Language Models (LLMs) in energy modeling to generate realistic, culturally sensitive, and behavior-specific data for household energy usage across diverse geographies. In this study, we employ and compare five different LLMs to systematically produce family structures, weather patterns, and daily consumption profiles for households in six distinct countries. A four-stage methodology synthesizes contextual daily data, including culturally nuanced activities, realistic weather ranges, HVAC operations, and distinct `energy signatures' that capture unique consumption footprints. Additionally, we explore an alternative strategy where external weather datasets can be directly integrated, bypassing intermediate weather modeling stages while ensuring physically consistent data inputs. The resulting dataset provides insights into how cultural, climatic, and behavioral factors converge to shape carbon emissions, offering a cost-effective avenue for scenario-based energy optimization. This approach underscores how prompt engineering, combined with knowledge distillation, can advance sustainable energy research and climate mitigation efforts. Source code is available at https://github.com/Singularity-AI-Lab/LLM-Energy-Knowledge-Distillation .