# Efficient Localness Transformer for Smart Sensor-Based Energy Disaggregation

arXiv: https://arxiv.org/abs/2203.16537 · HTML: https://mlsanity.com/paper/2203.16537/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 28 references that are in the map.
- First posted on arXiv 29 Mar 2022.

## Builds on (most cited first, 7 of 7)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Generating Long Sequences with Sparse Transformers](https://mlsanity.com/paper/1904.10509/) (2019)
- [Efficient Attention: Attention with Linear Complexities](https://mlsanity.com/paper/1812.01243/) (2018)
- [Neural NILM: Deep Neural Networks Applied to Energy Disaggregation](https://mlsanity.com/paper/1507.06594/) (2015)
- [Sequence-to-point learning with neural networks for nonintrusive load monitoring](https://mlsanity.com/paper/1612.09106/) (2016)
- [Subtask Gated Networks for Non-Intrusive Load Monitoring](https://mlsanity.com/paper/1811.06692/) (2018)

## Built on by (most cited first, 3 of 3)

- [Towards a Deeper Understanding of Transformer for Residential Non-intrusive Load Monitoring](https://mlsanity.com/paper/2410.03758/) (2024)
- [Scaled and Inter-token Relation Enhanced Transformer for Sample-restricted Residential NILM](https://mlsanity.com/paper/2410.12861/) (2024)
- [Ask for Any Appliance: A Prompt-Programmable Foundation Model for Non-Intrusive Load Monitoring](https://mlsanity.com/paper/2609.23146/) (2026)

## Abstract (arXiv)

Modern smart sensor-based energy management systems leverage non-intrusive load monitoring (NILM) to predict and optimize appliance load distribution in real-time. NILM, or energy disaggregation, refers to the decomposition of electricity usage conditioned on the aggregated power signals (i.e., smart sensor on the main channel). Based on real-time appliance power prediction using sensory technology, energy disaggregation has great potential to increase electricity efficiency and reduce energy expenditure. With the introduction of transformer models, NILM has achieved significant improvements in predicting device power readings. Nevertheless, transformers are less efficient due to O(l^2) complexity w.r.t. sequence length l. Moreover, transformers can fail to capture local signal patterns in sequence-to-point settings due to the lack of inductive bias in local context. In this work, we propose an efficient localness transformer for non-intrusive load monitoring (ELTransformer). Specifically, we leverage normalization functions and switch the order of matrix multiplication to approximate self-attention and reduce computational complexity. Additionally, we introduce localness modeling with sparse local attention heads and relative position encodings to enhance the model capacity in extracting short-term local patterns. To the best of our knowledge, ELTransformer is the first NILM model that addresses computational complexity and localness modeling in NILM. With extensive experiments and quantitative analyses, we demonstrate the efficiency and effectiveness of the the proposed ELTransformer with considerable improvements compared to state-of-the-art baselines.