Improved Electricity Price Forecasting Model Based on SHAP and Its Explainability Analysis

Background and Research Motivation

Electricity price forecasting (EPF) models have become a hot research topic in recent years, particularly due to the financial impact of market volatility on stakeholders. Especially in European energy markets, recent years have seen a sharp increase in fuel prices caused by the energy crisis and geopolitical tensions, leading to significant price volatility in electricity markets. Even a 1% prediction error can have substantial financial implications for stakeholders such as generation companies, load-serving entities, and trading companies. For instance, a mere 1% forecasting improvement could save a company with a 1GW load approximately $12 million annually. Hence, improving the accuracy of EPF models is critical for market participants.

Although machine learning (ML) and deep learning (DL)–based EPF models have shown significant accuracy improvements, the opaque “black-box” nature of these models limits stakeholder trust, especially during periods of abnormal price volatility. Explainable Artificial Intelligence (XAI) has emerged in recent years as an increasingly important tool to address this issue. However, most XAI methods are primarily confined to explaining model behavior or outputs and have not been fully explored in terms of their potential to improve model performance. This paper fills this research gap by proposing a SHAP (Shapley Additive Explanations)-based error compensation method that not only enhances model performance but also empowers non-technical users with simplified explanations to identify abnormal predictions.

Source and Author Information

The paper, titled “An Improved and Explainable Electricity Price Forecasting Model via SHAP-Based Error Compensation Approach,” was published in the January 2025 issue of IEEE Transactions on Artificial Intelligence (Vol. 6, No. 1). The study was conducted by researchers from academic and research institutions including Pandit Deendayal Energy University, Tallinn University of Technology, Sami Shamoon College of Engineering, and Technion Institute of Technology. The corresponding author is Leena Heistrene.

Research Approach and Methodology

The paper proposes a two-stage SHAP-based error compensation approach to improve EPF models. The first stage focuses on enhancing the performance of the base predictor model, while the second stage emphasizes user-friendly explanations that distinguish between regular and abnormal predictions. A detailed workflow is outlined below:

Stage 1: Error Quantification and Compensation

1. Base Predictor Selection and Training
   The base predictor model employs various machine learning techniques (e.g., LSTM, CNN, or XGBoost) to predict day-ahead electricity prices. Historical data from two different electricity markets (Italian and ERCOT markets) were used, training the model on input features such as historical prices, zonal prices, neighboring market prices, ancillary service prices, or system load.

2. Generating SHAP Explanations
   SHAP was applied to analyze the base predictor’s outputs, quantifying the impact of each feature on the model’s predictions via SHAP values (φ). These explanations provide local insights into individual prediction instances.

3. Designing the Corrector Model
   The study used SHAP values to identify error patterns in the base model’s predictions. A corrector model was trained using the SHAP-generated features (φsum and φxtrm) to estimate error compensation (εcomp). This compensation was then added to the base model’s predictions to produce the final prediction with improved accuracy.

Stage 2: Identifying Abnormal Predictions

The authors proposed a novel method to identify “regular predictions” and “abnormal predictions” using the SHAP scores output by the corrector model. Here’s how:

1. Defining Abnormal Predictions
   The authors introduced a z-score–based threshold (α) to identify predictions that deviate from the training data distribution. Predictions with |α| > 3 were classified as abnormal.

2. Analyzing SHAP Value Distributions
   By comparing SHAP explanations of regular and abnormal predictions, the study found that SHAP values for abnormal predictions show significant deviations and variations, allowing quick identification of unreliable forecasts. This feature enables easy assessment of prediction reliability, making it user-friendly for stakeholders.

Datasets and Experimental Setup

The study validated the proposed method on two real-world electricity markets with distinct characteristics (price volatility, market structures, and input feature sets):

1. Italian Electricity Market
   The target variable in this scenario was the “Pun” price, representing the uniform price of electricity in Italy. The training dataset comprised hourly data from 2015 to 2017, while data from 2018 were used to train the corrector model. Data from 2019 to 2021 were used for performance testing.

2. ERCOT Market
   The ERCOT market experiment focused on electricity prices in the Houston zone. This market provided a unique case with high price volatility, particularly during extreme market conditions like the unusual price spikes in August 2019 and February 2021.

The experiments utilized different model types (CNN, LSTM, XGBoost) to verify the generalizability of the approach.

Experimental Results and Analysis

Performance Improvement and Reliability

Through performance comparisons between the base predictor model and the compensation-enhanced model, significant reductions in RMSE, MAE, and MAPE were observed across different years and market scenarios. For example, in the Italian market’s 2021 dataset, MAPE was reduced from 5.73% to 3.23% under consistent data distribution conditions, and from 7.01% to 5.74% under distribution shift conditions.

Explainability of Abnormal Predictions

During events such as the Italian market’s 2021 fuel price surge and ERCOT market price spikes (e.g., August 2019), SHAP explanations from the corrector model successfully flagged abnormal predictions. The SHAP scores for abnormal predictions (e.g., φxtrm values) were significantly higher than those for normal predictions, demonstrating the method’s ability to identify irregular forecasts. The distinct change in SHAP rankings provided an intuitive way to understand unusual price behavior.

User-Friendly Explanations

Unlike the base model’s complex and feature-rich SHAP explanations, the corrector model relies on just three features, providing simplified insights suitable for non-technical users. For instance, power companies or trading platforms could quickly identify high-risk predictions using SHAP values, allowing them to optimize bidding strategies and reduce financial risks.

Contribution and Significance

1. Scientific Significance
   This study is the first to explore XAI techniques for improving the performance of time-series regression models, particularly EPF models. It opens a new avenue for expanding XAI applications.

2. Practical Relevance
   The proposed method’s generalizability, model-agnostic nature, and effectiveness in real-world markets make it a valuable tool for anomaly monitoring, risk assessment, and strategy optimization.

3. Innovative Contributions
   The SHAP-based error compensation method and the explanation framework for identifying abnormal predictions serve as pioneering efforts, providing a template for future research.

Future Outlook

The paper highlights the potential to extend this method to other forecasting tasks within the energy and power domains, such as electricity load forecasting or renewable energy generation prediction. Additionally, the anomaly detection tool could be integrated into automated bidding systems, enhancing decision-making efficiency in energy markets.