# Estimating the Unobservable Components of Electricity Demand Response with Inverse Optimization

arXiv: https://arxiv.org/abs/2410.02774 · HTML: https://mlsanity.com/paper/2410.02774/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 62 references that are in the map.
- First posted on arXiv 17 Sep 2024.

## Builds on (most cited first, 14 of 14)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Inverse Optimization: Theory and Applications](https://mlsanity.com/paper/2109.03920/) (2021)
- [Inverse Optimization with Noisy Data](https://mlsanity.com/paper/1507.03266/) (2015)
- [Data-driven Inverse Optimization with Imperfect Information](https://mlsanity.com/paper/1512.05489/) (2015)
- [A Data-driven Bidding Model for a Cluster of Price-responsive Consumers of Electricity](https://mlsanity.com/paper/1506.06587/) (2015)
- [Short-term Forecasting of Price-responsive Loads Using Inverse Optimization](https://mlsanity.com/paper/1607.07209/) (2016)
- [Forecasting the Price-Response of a Pool of Buildings via Homothetic Inverse Optimization](https://mlsanity.com/paper/2004.09819/) (2020)
- [Estimating Demand Flexibility Using Siamese LSTM Neural Networks](https://mlsanity.com/paper/2109.01258/) (2021)
- [Learning to Solve Bilevel Programs with Binary Tender](https://mlsanity.com/paper/2407.16914/) (2024)
- [A Contract Design Approach for Phantom Demand Response](https://mlsanity.com/paper/1611.09788/) (2016)
- [Data-Driven Demand-Side Flexibility Quantification: Prediction and Approximation of Flexibility Envelopes](https://mlsanity.com/paper/2110.12796/) (2021)
- [Baseline Estimation of Commercial Building HVAC Fan Power Using Tensor Completion](https://mlsanity.com/paper/2004.13557/) (2020)
- [Predicting Strategic Energy Storage Behaviors](https://mlsanity.com/paper/2306.11872/) (2023)

## Built on by (most cited first, 1 of 1)

- [Uncovering expert objectives in production planning via inverse optimization: An industrial case study](https://mlsanity.com/paper/2608.07398/) (2026)

## Abstract (arXiv)

Understanding and predicting the electricity demand responses to prices are critical activities for system operators, retailers, and regulators. While conventional machine learning and time series analyses have been adequate for the routine demand patterns that have adapted only slowly over many years, the emergence of active consumers with flexible assets such as solar-plus-storage systems, and electric vehicles, introduces new challenges. These active consumers exhibit more complex consumption patterns, the drivers of which are often unobservable to the retailers and system operators. In practice, system operators and retailers can only monitor the net demand (metered at grid connection points), which reflects the overall energy consumption or production exchanged with the grid. As a result, all "behind-the-meter" activities-such as the use of flexibility-remain hidden from these entities. Such behind-the-meter behavior may be controlled by third party agents or incentivized by tariffs; in either case, the retailer's revenue and the system loads would be impacted by these activities behind the meter, but their details can only be inferred. We define the main components of net demand, as baseload, flexible, and self-generation, each having nonlinear responses to market price signals. As flexible demand response and self generation are increasing, this raises a pressing question of whether existing methods still perform well and, if not, whether there is an alternative way to understand and project the unobserved components of behavior. In response to this practical challenge, we evaluate the potential of a data-driven inverse optimization (IO) methodology. This approach characterizes decomposed consumption patterns without requiring direct observation of behind-the-meter behavior or device-level metering [...]