Ensemble-Based Peak Demand Probability Density Forecasting with Application to Risk-Aware Power System Scheduling
Extracted source text · arXiv 2506.01358
\documentclass[titlepage]{elsarticle}
\usepackage[section]{placeins}
\usepackage{textcomp}
\usepackage{gensymb}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsmath}
\usepackage{glossaries}
\usepackage{siunitx}
\usepackage{subcaption}
\usepackage{hyperref}
\renewcommand\figureautorefname{Fig.}
\renewcommand\tableautorefname{Table}
\renewcommand\sectionautorefname{Section}
\renewcommand\subsectionautorefname{Subsection}
\def\equationautorefname~#1\null{eq. (#1)\null}
\newacronym{EVA}{EVA}{extreme value analysis}
\newacronym{PDF}{PDF}{probability density function}
\newacronym{CDF}{CDF}{cumulative distribution function}
\newacronym{GEV}{GEV}{generalized extreme value distribution}
\newacronym{GPD}{GPD}{generalized Pareto distribution}
\newacronym{KL}{KL}{Kullback–Leibler divergence}
\newacronym{MLE}{MLE}{maximum likelihood estimate}
\newacronym{CART}{CART}{classification and regression trees}
\newacronym{PWM}{PWM}{probability weighted moments}
\newacronym{iid}{i.i.d.}{independently and identically distributed}
\newacronym{DSM}{DSM}{demand-side management}
\newacronym{VaR}{VaR}{value-at-risk}
\newacronym{CVaR}{CVaR}{conditional value-at-risk}
\newacronym{EUE}{EUE}{expected unserved energy}
\newacronym{LOLP}{LOLP}{loss-of-load probability}
\newacronym{NERC}{NERC}{North American Electric Reliability Corporation}
\newacronym{CNN}{CNN}{convolution neural network}
\newacronym{QTCN}{QTCN}{quantile temporal convolution neural network}
\newacronym{NGD}{NGD}{natural gradient descent}
\newacronym{CRB}{CRB}{Cram\'er-Rao bound}
\newacronym{DOA}{DOA}{domain of attraction}
\newacronym{QRF}{QRF}{quantile regression forest}
\newacronym{LPHC}{LPHC}{low-probability high-consequence}
\newacronym{PPM}{PPM}{parts per million}
\newacronym{AI}{AI}{artificial intelligence}
\newacronym{HIA}{HIA}{hourly integrated average}
\newacronym{ML}{ML}{machine learning}
\newacronym{QR}{QR}{quantile regression}
\newacronym{ANN}{ANN}{artificial neural network}
\newacronym{KDE}{KDE}{kernel density estimation}
\newacronym{QF}{QF}{quantile function}
\newacronym{ACF}{ACF}{autocorrelation function}
\newacronym{MSE}{MSE}{mean squared error}
\newacronym{MVUE}{MVUE}{minimum-variance unbiased estimator}
\newacronym{RTO}{RTO}{regional transmission organization}
\newacronym{SCED}{SCED}{security-constrained economic dispatch}
\newacronym{LMP}{LMP}{locational marginal price}
\newacronym{MCP}{MCP}{market clearing price}
\newacronym{RAC}{RAC}{reliability assessment and commitment}
\newacronym{DASC}{DASC}{day-ahead scheduling capacity}
\newacronym{MAE}{MAE}{mean absolute error}
\newacronym{FOR}{FOR}{forced outage rate}
\newacronym{DAEM}{DAEM}{day-ahead energy market}
\newacronym{GAMLSS}{GAMLSS}{generalized additive model for location, scale and shape}
\newacronym{LOLE}{LOLE}{loss-of-load expectation}
\newacronym{AEMO}{AEMO}{Australian Energy Market Operator}
\newacronym{LSTM}{LSTM}{long short-term memory}
\newacronym{GARCH}{GARCH}{generalized autoregressive conditional heteroskedasticity}
\newacronym{USE}{USE}{unserved energy}
\newacronym{EENS}{EENS}{expected energy not served}
\newacronym{CRPS}{CRPS}{continuous ranked probability score}
\begin{document}
\title{Prediction of the Conditional Probability Densities of Time Interval Extrema with Application to Risk-Sensitive Scheduling\tnoteref{funding}\tnoteref{source-code}}
\tnotetext[funding]{This paper is partly derived from the work supported by the U.S. Department of Energy’s Office of Energy Efficiency and Renewable Energy (EERE) through the Solar Energy Technology Office (SETO) under Award DE-EE0009357. The funding source has no involvement in the development of the methodology, design, preparation, or interpretation of the case study, or the decision to submit for publication. }
\tnotetext[source-code]{Source code artifacts are available in repository \url{https://github.com/byu9/extreme-tree}. Results in the paper correspond to revision \texttt{6e80006}.}
\author[ncsu]{Buyi Yu}\corref{cor1}\ead{byu9@ncsu.edu}
\author[ncsu]{Wenyuan Tang}\ead{wtang8@ncsu.edu}
\cortext[cor1]{Corresponding author}
\affiliation[ncsu]{
addressline={Campus Box 7571},
organization={North Carolina State University},
city={Raleigh, NC 27695},
country={Untied States}
}
\input{001-abstract}
\maketitle
\input{002-introduction}
\input{003-problem}
\input{004-methodology}
\input{005-example}
\input{006-case-study}
\input{007-conclusion}
\input{008-acknowledgements}
\input{009-ai-declaration}
\bibliographystyle{elsarticle-num}
\bibliography{900-references}
\end{document}
\begin{abstract}
Planning and scheduling activities in the electrical power system, such as the commitment of reserve generation, often involve the statistical characterization of peak demand.
Due to the stationarity assumption of classical \gls{EVA}, existing approaches in the industry apply \gls{EVA} on simulated annual peaks created by weather-dependent surrogate models using Mont\'e-Carlo simulations on a per-scenario basis.
In day-ahead scheduling, the daily peak demand changes upon various factors besides temperature, Mont\'e-Carlo experiments become intractable, and state-of-the-art \gls{GAMLSS}-based nonstationary \gls{EVA} is often impractical due to convergence issues on high-dimensional covariates.
This article explores uncharted territories and proposes a novel nonstationary \gls{EVA} estimator that predicts the probable peaks of high-resolution time intervals and their corresponding conditional probability densities based on calendar information and weather conditions where historical peaks are observed. Compared to \gls{GAMLSS}, our method automatically discovers and robustly models complex relationships between the covariate and the peak demand density.
We present a case study on the determination of day-ahead scheduling capacity and demonstrate that compared to the industry approach, our approach results in a $38\%$ reduction in the yearly total committed capacity while maintaining the given risk requirement.
\end{abstract}
\begin{keyword}
Risk-averse unit commitment\sep
extreme value analysis\sep
ensemble model\sep
estimation theory\sep
probabilistic regression\sep
energy market
\end{keyword}
\section{Introduction}
Increasingly frequent extreme weather events, such as the winter storm that resulted in the 2021 Texas power crisis, can disrupt power balance, leading to cascading failures of the power grid with devastating and sustained impacts \cite{flores_2021_2023}.
From long-term planning to intra-hour real-time operation, rigorous efforts are dedicated to characterizing extreme capacity shortage events and the statistical analysis from which resource adequacy requirements are partly determined.
For example, according to the 2016 technical guideline document \cite{nerc_probabilistic_2016}, the \gls{NERC} leverages cost-effectiveness against risk acceptability and performs annual probabilistic assessments that cover worst-case probabilistic evaluations of generation outages, transmission contingencies, availability of renewable and \gls{DSM} resources, and demand uncertainty.
In the ISO New England 2027 winter study \cite{george_stephen_preliminary_2023}, probabilistic extrema analysis is utilized to identify high-risk weather scenarios that may result in severe energy shortfalls. Probabilistic predictions of generation deficits in the form of survival functions and \glspl{PDF} have also been introduced as stochastic risk control constraints \cite{dillon_integer_1978} \cite{shi_ant_2004}.
Under increasing penetration of utility-scale and behind-the-meter variable renewable resources, literature on energy forecasting indicates recognition of the practicality of probabilistic predictions, also known as full-distribution metrics, in mitigation of the effects of resource intermittency and variability to system operations \cite{haupt_use_2019,impram_challenges_2020,sweeney_future_2020}.
\subsection{Characteristic and requirements of risk-sensitive scheduling}
Secure power system operations must compensate for generator outages, demand fluctuations, and the uncertainty of increasingly common renewable energy systems. We use risk-sensitive scheduling to refer to operator decisions that explicitly account for the risk of high-impact events characterized by their probability and severity using statistical methods. Such decisions straddle the balance between the increased cost of resource over-commitment, which results in waste of natural resources, and the consequence of under-commitment, such as the societal impact of blackouts.
Contrary to economic unit commitment, which bases decisions on the typical demand, risk-sensitive security-constrained unit commitment decisions include modeling of the peak demand since system failures are caused by extremes. Given the ethical responsibility of the system operator as a regulated monopoly entrusted with a mission-critical infrastructure, energy regulatory and oversight agencies create stringent reliability guidelines to enforce accountability, and these guidelines are often given in terms of probabilistic measures formulated on the statistical behavior of peak demand. For example, in North America, \gls{NERC} standard BAL-502-RF-03 \cite{north_american_electric_reliability_corporation_bal-502-rf-03_2017} specifies reliability requirement in terms of \gls{LOLP}, defined as the probability that the demand exceeds available generation capacity; in Australia, the \gls{AEMO} specifies reliability requirement in National Electricity Law \cite{daniel_cornelis_van_holst_pellekaan_draft_nodate} in terms of \gls{EUE}, also known simply as the \gls{USE} or \gls{EENS}, defined as the statistical expectation of the amount of energy the system cannot serve due to demand exceeding capacity, often given in percentage of annual total energy served; in the European Union, the regulation (EU) 2019/943 \cite{noauthor_regulation_2019} demands transparency and regular monitoring of the \gls{EUE} and \gls{LOLE}, defined as the statistical expectation of the number of time intervals in which the demand exceeds available generation capacity.
While the aforementioned regulatory risk metrics currently only apply to annual time intervals, our approach provides the necessary means to model risk in much shorter intervals to allow fine-grained control. This capability is increasingly important for the management of short-term intermittency and volatility of high-penetration renewable energy systems.
\subsection{Challenges with existing forecasting approaches}
Probabilistic regression is a method that estimates likelihoods from deterministic observations.
In parametric probabilistic regression, the probable outcome $y(\boldsymbol{x})$ under the condition known as the covariate $\boldsymbol{x}$ is assumed to follow a predetermined distribution family $D(\hat{\boldsymbol\theta}(\boldsymbol {x}))$ with predicted deterministic parameter $\hat{\boldsymbol \theta}(\boldsymbol{x})$.
In non-parametric probabilistic regression, the density does not assume a predetermined form. Compared to parametric probabilistic regression, non-parametric probabilistic regression is largely assumption-free and excels at prediction tasks where the distribution family is unknown, while parametric probabilistic regression requires the establishment of statistical assumptions that may be unsupported by observational evidence.
Popular non-parametric probabilistic regression methods, such as the well-known \gls{QR} \cite{koenker_regression_1978} and recent \gls{ANN}-based quantile forecasting methods such as \gls{CNN} \cite{jiotsop-foze_electrical_2024}, attentive \gls{QTCN} \cite{guo_probabilistic_2024}, and deep reinforcement learning \cite{li_probabilistic_2024} produce estimates of the \gls{QF}, also known as the inverse \gls{CDF} on a set of quantiles.
Despite their popularity, quantile forecasting methods have several notable shortcomings. First, they tend to suffer from quantile crossover \cite{waldmann_quantile_2018}, referring to the phenomenon that the approximated inverse \gls{CDF} is not in a monotonically non-decreasing form, implying the existence of unsubstantial negative densities.
Second, quantile forecasting methods produce discrete estimates of the probability density on a limited set of quantiles. To obtain a continuous density that can be adopted by stochastic optimal scheduling problems utilizing gradient-based optimization, the predicted quantiles must undergo post-processing such as \gls{KDE} \cite{rosenblatt_remarks_1956} to interpolate the missing quantiles.
Last, as we will demonstrate through an experiment later in this paper, due to the rarity of observations at extreme quantiles, quantile forecasting methods are unable to approximate the probability density of \gls{LPHC} events, also known as ``black swan" events.
As an alternative to non-parametric methods, parametric probabilistic regression methods such as \gls{GARCH} \cite{engle_autoregressive_1982, bollerslev_generalized_1986} and \gls{GAMLSS} \cite{rigby_generalized_2005} have been proposed. While \gls{GARCH} can model nonstationarity of the variance, also known as the second moment, \gls{GAMLSS} can account for the nonstationarity of the location, scale, and shape parameters by modeling them as functions of the covariate using explicit formulas or spline approximations. However, existing literature such as \cite{mayr_gamlss_2010} has described \gls{GAMLSS} to struggle with high-dimensional covariates, and in our case study in \autoref{sec:case_study}, \gls{GAMLSS} failed to converge.
\subsection{Accurately estimating the density of extreme values}
Time series often contain far more ordinary observations than extreme observations. As a result, regardless of parametric or non-parametric probabilistic regression, directly estimating the probability density of rare occurrences, although straightforward, does not provide well-calibrated density estimates of the outliers and tends to bias towards the accuracy of the inliers.
The density of extrema can be approximated more accurately if the time series only contains extreme observations, which can be obtained using two common approaches.
The block maxima approach divides a high-resolution time series into lower-resolution and non-overlapping time intervals, each containing a relatively large number of observations, before taking the maximum of each interval.
Alternatively, the points-over-threshold approach retains extreme observations above a threshold and drops all other observations. The block minima and points-under-threshold can be obtained in similar ways.
\Gls{EVA} is a mature branch of statistics that studies the accurate modeling of the probability density of the outliers of various statistical distributions and has been the cornerstone in the prediction of rare natural disasters such as fire, flood, and earthquake, reliability studies of materials and aerospace equipment, and finance \cite{bousquet_extreme_2021}. It has been discovered that as the block size increases, if the block maxima of \gls{iid} observations of an unknown distribution converge, it will approach asymptotically to the \gls{GEV}. Similarly, as the threshold increases, if the distribution of points-over-threshold converges, it will approach asymptotically to the \gls{GPD}. Due to symmetry, the same phenomenon is demonstrated on the block minima and points-under-threshold and applies to sign-opposite transformations of the \gls{GEV} and \gls{GPD}. Despite the lack of observations at extreme quantiles, given a sufficient block size or threshold, the probability density of extrema can be accurately predicted in the form of \gls{GEV} or \gls{GPD}.
While \gls{GEV} or \gls{GPD} used in other engineering fields often model some form of nonstationarity in the location and scale parameters, fitting of nonstationary models often involves manual selection of trend models \cite{renard_use_2007}. In the case of \gls{GEV}, the shape parameter is considered to be more difficult to estimate, and engineering methods tend to assume a constant shape parameter \cite{katz_statistics_2002, martins_generalized_2000}. The method proposed herein can automatically discover the potentially nonlinear relationship between a high-dimensional covariate and all three \gls{GEV} parameters, including the shape.
\subsection{Reconciling diverging trends in forecasting research and the industry}
Despite risk management concentrating on controlling the occurrence and reducing the severity of the worst case, recent literature tends to favor quantile forecasting methods, which are typically optimized for overall performance, overlooking the rare extremes where critical system risks lie.
A recent report from the industry \cite{ming_role_2024} signaled improved probabilistic modeling as one of the critical forward-facing issues in ensuring reliability.
Whereas classical \gls{EVA} is the canonical approach in extreme value probabilistic assessments, it assumes stationarity and cannot model the effect of external factors on the estimated density, preventing the adoption of classical \gls{EVA} in high-resolution applications such as day-ahead scheduling, whereas the state-of-the-art \gls{GAMLSS} is often impractical due to the aforementioned convergence issues.
Leveraging established machine learning techniques and \gls{EVA}, the paper proposes a novel method to model and predict peak demand uncertainty. The objectives of this paper are as follows:
\begin{itemize}
\item To develop a widely applicable conditional parametric density model for time series extrema that surpasses quantile-based models in accurately modeling ``black swan" events.
\item To propose a novel nonstationary \gls{EVA} estimator based on the proposed density model and overcome the stationarity limitation of classical \gls{EVA} and the convergence issue of \gls{GAMLSS}.
\item To apply nonstationary \gls{EVA} in high-resolution risk-sensitive scheduling applications in the power system.
\end{itemize}
While many existing forecasting methods, such as the aforementioned deep learning methods \cite{jiotsop-foze_electrical_2024, guo_probabilistic_2024, li_probabilistic_2024}, focus on predicting point values or quantiles, our approach aims to estimate the conditional distribution of peak demand using an \gls{EVA} framework. We use \autoref{tab:distinguishing_features_of_our_method} to summarize the key differences that distinguish our method from a large portion of existing forecasting literature.
\begin{table}[htb!]
\centering
\scalebox{0.9}{
\begin{tabular}{lll}
\hline
Aspect & Existing methods & Proposed method \\
\hline
Prediction format & Series of mean or quantiles & Series of probability densities \\
Forecasting target & Ordinary and peak values & Only peak densities \\
Accuracy of peaks & Often overlooked & Specifically targeted \\
Assumptions & Often empirical & Conditional EVA framework \\
Reliability metrics & Often limited to quantiles & VaR, CVaR, EUE, LOLP \\
\hline
\end{tabular}
}
\caption{Key differences that distinguish our method from existing forecasting literature. EVA---extreme value analysis, VaR---value-at-risk, CVaR---conditional value-at-risk, EUE---expected unserved energy, LOLP---loss-of-load probability.}
\label{tab:distinguishing_features_of_our_method}
\end{table}
\subsection{Key contributions and paper structure}
The key contributions of this paper are, first, the formulation of a rigorous statistical framework to model nonstationary time series extrema, second, a novel nonstationary \gls{EVA} approach to estimate \gls{GEV} parameters under the proposed framework, and finally, a case study that demonstrates a realistic use case of the proposed approach. The structure of the paper and minor contributions are as follows:
\begin{itemize}
\item In the next section, we describe the problem, formulate the conditional framework to model nonstationary extrema densities, elucidate the assumptions, investigate the conditions for the weakening of the framework, and present two configurations to model autocorrelation.
\item In \autoref{sec:methodology}, we conceive and expound a novel nonstationary \gls{EVA} approach, compare two estimation methods, justify the selection, and elaborate on the step-by-step operation of the algorithm and its termination.
\item In \autoref{sec:example}, through an experiment, we contrast our approach with a theoretically most efficient estimator, juxtapose it with two competitors, scrutinize the results, and corroborate the superiority of our approach.
\item In \autoref{sec:case_study}, through a case study, we integrate our approach with a real-world application, augment existing industry practices, and assess the benefits of risk management and cost reduction.
\end{itemize}
Given potential criticisms regarding the lack of deep learning in the proposed approach, we acknowledge that deep learning models, such as \gls{LSTM} networks \cite{hochreiter_long_1997} or temporal convolutional networks \cite{bai_empirical_2018}, can be highly effective at capturing long-term dependencies in time series data. However, the objective of this paper is not to propose yet another deep-learning method on the well-studied topic of deterministic regression, but rather a novel probabilistic regression method that estimates the time-varying probability densities of peak demand using a structured \gls{EVA} framework. While we considered deep learning during the design of our method, it is not naturally aligned with established \gls{EVA} estimators. We view the integration of deep learning as an interesting direction for future exploration, but not as a necessary condition for establishing the value and practicality of our method in its current form.
\section{Problem statement}
\label{sec:problem_statement}
\begin{figure}[htb!]
\centering
\includegraphics[width=0.8\textwidth]{999-fig-001.pdf}
\caption{Sub-interval observations, interval maxima, and the predicted conditional probability density of the interval maximum.}
\label{fig:model_interval}
\end{figure}
For deterministic sub-interval condition $\boldsymbol{x}$, we use symbol $\mathcal{Y}$ to represent the event that the peak demand of the enclosing interval is observed at $\boldsymbol{x}$. Given the knowledge that event $\mathcal{Y}$ occurs, our goal is to predict the distribution of the peak demand $\Pr(y|\mathcal{Y}, \boldsymbol{x})$ based on future environmental condition $\boldsymbol{x}$, which is also known and deterministic. The phrase ``peak demand'' implies that it is an observation that belongs to a time interval. The relationship between sub-interval observations, intervals, and interval maxima is illustrated in \autoref{fig:model_interval}. Using $\Pr(\mathcal{Y}|\boldsymbol{x})$ to represent the probability of sub-interval condition $\boldsymbol{x}$ resulting in peak demand event $\mathcal{Y}$, the joint probability $\Pr(y, \mathcal{Y} | \boldsymbol{x})$ of peak demand event $\mathcal{Y}$ of severity $y$ occurring under sub-interval condition $\boldsymbol{x}$ is given by
\begin{equation}
\Pr(y, \mathcal{Y} | \boldsymbol{x}) = \Pr(y|\mathcal{Y}, \boldsymbol{x}) \Pr(\mathcal{Y} | \boldsymbol{x}).
\label{eq:product_rule}
\end{equation}
While it is known as a rule-of-thumb that daily peaks tend to be observed under hot weather and in the afternoon, estimation of peak demand event frequency $\Pr(\mathcal{Y} | \boldsymbol{x})$ is beyond the scope of this paper. When $\Pr(\mathcal{Y} | \boldsymbol{x})$ is unknown, in risk management it is prudent to assume the worst-case using $\Pr(\mathcal{Y} | \boldsymbol{x}) \equiv 1$, where peak demand events are assumed to always occur at every sub-interval and our predictions $\hat{\Pr}(y|\mathcal{Y}, \boldsymbol{x})$ directly yields the response joint distribution $\Pr(y, \mathcal{Y} | \boldsymbol{x})$. The identity \autoref{eq:product_rule} may invoke the context of a Bayesian method and create an impression of attempts to conceal its flaws. We therefore wish to emphasize that the method introduced herein is not a Bayesian method, \autoref{eq:product_rule} is always evaluated from right to left, and we do not use Bayes' rule to find the probability of a cause given its effect. The choice of applying $\Pr(\mathcal{Y} | \boldsymbol{x})\equiv 1$ is justified as described.
\begin{figure}[htb!]
\centering
\includegraphics[width=\textwidth]{999-fig-002.pdf}
\caption{Example use case. The predictions are in the form of conditional daily peak distribution $y \sim \text{GEV}(\hat{\boldsymbol{\theta}}|\boldsymbol{x})$ that is updated at every hour. The covariate elements and results in the artwork are examples intended to illustrate the mechanics of our method and not to be taken as prescriptive implementation advice. GEV---generalized extreme value distribution.}
\label{fig:model_overview}
\end{figure}
A typical application of our approach is shown in \autoref{fig:model_overview}. The covariates exemplified in the artwork reflect common types of elements often chosen in time-series forecasting, and the actual covariate structure is application-specific and informed by domain knowledge. We invite the reader to skip ahead to the case study in \autoref{sec:case_study}, which demonstrates the choice of covariate for a real-world application. For the set of covariate dimensions $\mathcal{M}$, set of hourly intervals $\mathcal{N}$, and supplied covariate $\boldsymbol{x}_{\mathcal{M},\mathcal{N}}$ corresponding to hourly conditions, we produce estimates of the conditional daily peak distribution $\hat{\Pr}(y|\mathcal{Y}, \boldsymbol{x}) = \text{GEV}(\hat{\boldsymbol{\theta}}|\boldsymbol{x})$ at every hour.
\subsection{Conditional framework for nonstationary extreme value densities}
For unknown initial distribution $\text{D}(\boldsymbol{x})$ that may change with covariate $\boldsymbol{x}$, sub-interval observations $z(\boldsymbol{x})$ under condition of $\boldsymbol{x}$ are assumed to be \gls{iid} realizations of the following stochastic process:
\begin{equation}
z(\boldsymbol{x}) \sim \text{D}(\boldsymbol{x}) \label{eq:assumption}
\end{equation}
For the block maxima $ y(\boldsymbol{x}) =\max(z_1, \cdots, z_M|\boldsymbol{x})$ of $M$ \gls{iid} observations, according to the Fisher-Tippett-Gnedenko theorem, if as $M \rightarrow \infty$ the distribution of $y(\boldsymbol{x})$ is non-degenerate, it is well approximated by the \gls{GEV}, regardless of the initial distribution $D(\boldsymbol{x})$.
The \gls{DOA} refers to the set of initial distributions whose block maxima can be approximated by \gls{GEV}. For $p_1, \cdots, p_4\in(0,1)$, the necessary and sufficient condition for initial distribution $D$ with inverse \gls{CDF} $F^{-1}(z)$ to belong to the \gls{DOA} is \cite{castillo_extreme_1988}:
\begin{equation}
\lim_{M\to\infty} \frac{F^{-1}(p_1^{1/M})-F^{-1}(p_2^{1/M})}{F^{-1}(p_3^{1/M})-F^{-1}(p_4^{1/M})} =
\frac{(-\log p_1)^{-\xi} - (-\log p_2)^{-\xi}}{(-\log p_3)^{-\xi} - (-\log p_4)^{-\xi}}
\end{equation}
The \gls{DOA} of \gls{GEV} encompasses a wide range of initial distribution families, making the statistical framework widely applicable. In general, block maxima of initial distributions with exponentially or sub-exponentially decaying upper tails, such as normal, exponential, $\log$-normal, Gamma, and Rayleigh, coverage to the type-I \gls{GEV} known as Gumbel, where $\xi=0$, those with polynomial decaying upper tails such as Cauchy, Pareto, and Student's $t$ converge to the type-II \gls{GEV} known as Fr\'echet, where $\xi>0$, and those with a finite upper bound, such as uniform or beta, converge to the type-III \gls{GEV} known as inverse Weibull, where $\xi<0$.
For probable outcome $y$, location $\mu \in \mathbb{R}$, scale $\sigma > 0$, and shape $\xi \in \mathbb{R}$, the support of \gls{GEV} refers to $y$ that satisfies $1 + \xi \left( \frac{y-\mu}{\sigma} \right) > 0$. The \gls{PDF} of the \gls{GEV} is given by
\begin{equation}
\label{eq:gev_pdf}
\Pr_\text{GEV}(y;\mu,\sigma,\xi) = \frac{1}{\sigma} \tau^{1+\xi} \exp(-\tau),
\end{equation}
where
\begin{equation}
\tau(y;\mu,\sigma,\xi) =
\begin{cases}
\left(1 + \xi \frac{y- \mu}{\sigma} \right)^{-\frac{1}{\xi}} &
\xi \neq 0, \\
\exp \left( - \frac{y - \mu}{\sigma} \right) & \xi = 0.
\end{cases}\label{eq:tau}
\end{equation}
When the block size $M$ is sufficiently large, the block maxima and minima can be modeled by
\begin{subequations}
\begin{align}
\label{eq:max_follows_gev}
y^\text{MAX}(\boldsymbol{x}) &\sim \text{GEV}(\mu^{\text{MAX}}, \sigma^{\text{MAX}}, \xi^{\text{MAX}}|\boldsymbol{x}), \\
\label{eq:min_follows_gev}
-y^\text{MIN}(\boldsymbol{x}) &\sim \text{GEV}(-\mu^{\text{MIN}}, \sigma^{\text{MIN}}, \xi^{\text{MIN}}|\boldsymbol{x}),
\end{align}\label{eq:max_min_follows_gev}
\end{subequations}
where
\begin{subequations}
\begin{align}
y^\text{MAX}(\boldsymbol{x}) &= \max(z_1, \cdots, z_K|\boldsymbol{x}), \\
y^\text{MIN}(\boldsymbol{x}) &= \min(z_1, \cdots, z_K|\boldsymbol{x}).
\end{align}
\end{subequations}
\subsection{Addressing autocorrelation}
\label{sec:autocorrelation}
When improperly modeled, autocorrelation in time series results in the weakening of the proposed framework due to violation of the \gls{iid} assumption \autoref{eq:assumption}.
As discussed in \cite{majumdar_extreme_2019}, \gls{GEV} is still applicable under weak autocorrelation as long as the block maxima are effectively independent. Furthermore, it is theoretically proven in \cite{leadbetter_extremes_1983} that even with relatively strong autocorrelation with a slow rate of decay of the \gls{ACF}, the classical asymptotic behavior of \gls{EVA} still holds.
For example, \gls{GEV} can be applied to predict the probability of extreme floods in hydrology based on monthly rainfall maxima, despite the month-to-month correlation. Examples of \gls{EVA} being applied to potentially weakly autocorrelated series in power system applications include \cite{ganger_statistical_2014}, where autocorrelation in the ramp rate is reduced due to differencing, and \cite{wang_high_2018}, where the peaks are effectively independent. We argue that the conditional \gls{EVA} framework we established in \autoref{eq:max_min_follows_gev} is still applicable under strong autocorrelation using either of the following configurations:
\begin{itemize}
\item Direct modeling of autocorrelation: For vector $\boldsymbol{z}_{<n}=[z_{n-r},\cdots, z_{n-2}, z_{n-1}]^\intercal$ representing the history of $z_n$ up to lag order $r$, assuming that strongly correlated lag orders are included in $\boldsymbol{z}_{<n}$, for exogenous factors $\boldsymbol\psi_n$, an autoregressive nonstationary process can be approximated as locally \gls{iid} using the unknown conditional distribution $z_n \sim \text{D}(z|\boldsymbol\psi_n,\boldsymbol{z}_{<n})$, which is in the form of \autoref{eq:assumption}. An example of this configuration is shown in \autoref{fig:model_overview}, where the predicted daily peak distribution is conditioned on strongly correlated historical demand.
\item Delegated modeling of autocorrelation: For $\hat{z}_n$ representing the forecast produced by a base regression model known as the delegate, given the assumption that the residuals of such model are non-autocorrelated, for exogenous factors $\boldsymbol\psi_n$, an autoregressive nonstationary process can be approximated as locally \gls{iid} using the unknown conditional distribution $z_n \sim \text{D}(z|\boldsymbol\psi_n, \hat{z}_n)$, which is in the form of \autoref{eq:assumption}. This configuration is recommended when the delegate is already highly accurate, allowing our approach to take advantage of the delegate when creating the peak demand probabilistic forecast. An example of this configuration is the PJM case study in \autoref{sec:case_study}.
\end{itemize}
We will demonstrate through the case study in \autoref{sec:case_study} that under worst-case baseline probability, this conditional \gls{EVA} framework allows us to calculate the exact amount of required day-ahead capacity to satisfy a given risk requirement and, compared to annual \gls{EVA} assessments, prevents over-commitment of generation resources and can potentially lead to a significant reduction of wholesale electricity prices.
\section{Methodology}
\label{sec:methodology}
Contrary to single-valued forecasting approaches such as least-squares regression, which aims to minimize the \gls{MSE} of predictions against the corresponding targets, given a set of historical conditions and block maxima in the form of covariate--target pairs $(\boldsymbol{x}_{\mathcal{M}\mathcal{N}}, y_{\mathcal{N}})$, the objective of the proposed approach is to predict the parameters of the conditional distribution $\text{GEV}(\hat{\boldsymbol{\theta}}_{\mathcal{N}}|\boldsymbol{x}_{\mathcal{M},\mathcal{N}})$.
We adopt a \gls{ML} concept known as the bootstrap aggregating ensemble \cite{breiman_bagging_1996}, also known as bagging ensemble, which consists of $K$ conditional estimators, each in the form of a decision tree fitted on subsets of the covariate--target pairs using a procedure we proposed. The purpose of the ensemble, discussed in later sections, is to improve the estimate stability and statistical efficiency. The fitting process of the ensemble is illustrated in \autoref{fig:ensemble_fitting}. The subsets are randomly sampled from the parent dataset with replacement using a procedure known as bootstrapping, where the ratio $\rho \in (0, \infty)$ controls the size of the subsets relative to their parent. Due to replacement, it is possible to create subsets larger than the original dataset. During inference, as shown in \autoref{fig:model_overview}, the estimates $\hat{\boldsymbol{\theta}}'_{k,\mathcal{N}} (\boldsymbol{x}_{\mathcal{M}, \mathcal{N}})$ produced by members $k=1, \cdots, K$ are averaged in the member dimension to create the final estimate $\hat{\boldsymbol{\theta}}_\mathcal{N}(\boldsymbol{x}_{\mathcal{M}, \mathcal{N}})$:
\begin{equation}
\hat{\boldsymbol{\theta}}_\mathcal{N}(\boldsymbol{x}_{\mathcal{M}, \mathcal{N}}) = \frac{1}{K} \sum_{k=1}^K \hat{\boldsymbol{\theta}}'_{k,\mathcal{N}} (\boldsymbol{x}_{\mathcal{M}, \mathcal{N}})
\end{equation}
\begin{figure}[htb!]
\centering
\includegraphics[width=0.9\textwidth]{999-fig-003.pdf}
\caption{Ensemble fitting through bootstrapping, also known as resampling with replacement.}
\label{fig:ensemble_fitting}
\end{figure}
\subsection{GEV parameter estimation}
A well-known metric to quantify the error of probabilistic forecast $\Pr_A(y;\hat{\boldsymbol \theta}_A)$ of distribution family $A$ against a reference probabilistic forecast $\Pr_B(y;\boldsymbol \theta_B)$ of distribution family $B$ is the \gls{KL} \cite{kullback_information_1951}.
For observed outcome $y_n$, the ideal probabilistic forecast is a degenerate distribution that predicts $y_n$ with infinite probability density denoted by the Dirac delta function $\delta(\cdot)$.
As shown in \autoref{eq:kl_divergence}, simplification of \gls{KL} against a degenerate distribution $B$ where $\Pr_B(y;\boldsymbol{\theta}_B)=\delta(y-y_n)$ leads to the log score, whose minimization results in the \gls{MLE}:
\begin{equation}
\label{eq:kl_divergence}
\begin{aligned}
\text{KL}(\hat{A}, B) &=
\int \Pr_B(y;\boldsymbol \theta_B) \log \left(\frac{\Pr_B(y;\boldsymbol \theta_B)}{\Pr_A(y;\hat{\boldsymbol \theta}_A)} \right) dy \\
&= \int \delta(y-y_n) \log \delta(y-y_n) dy -\int \delta(y-y_n) \log \Pr_A(y;\hat{\boldsymbol \theta}_A) dy \\
= & \left( \log \delta(x) \int \delta(x)dx - \int \frac{d}{dx} \log\delta(x) dx \right) -\log \Pr_A(y_n;\hat{\boldsymbol \theta}_A) \\
&= -\log \Pr_A(y_n;\hat{\boldsymbol \theta}_A)
\end{aligned}
\end{equation}
The \gls{MLE} for \gls{GEV} does not have an explicit mathematical representation and the procedure to compute it involves an iterative process starting from an initial estimate, guided by the gradient of the log score using a method now widely known as \gls{NGD} \cite{amari_natural_1998}.
For \gls{GEV} Fisher information matrix $\boldsymbol{I}(\hat{\boldsymbol{\theta}})$ and (ordinary) gradient $\nabla L$ of the \gls{GEV} log score, the natural gradient $\tilde{\nabla} L$ is given by \cite{duan_ngboost_2020}:
\begin{equation}
\tilde{\nabla} L(y_n;\hat{\boldsymbol{\theta}}) =
\boldsymbol{I}^{-1} (\hat{\boldsymbol{\theta}})
\cdot
\nabla L(y_n;\hat{\boldsymbol{\theta}}),
\label{eq:natural_gradient}
\end{equation}
where $\tau$ is shown in \autoref{eq:tau}.
For \gls{GEV}, the log score is given by
\begin{equation}
L (y_n; \boldsymbol{\theta}) = \sigma - (1 + \xi) \log \tau + \tau,
\label{eq:log_score}
\end{equation}
and its (ordinary) gradient is given by \cite{dey_extreme_2016}
\begin{subequations}\label{eq:ordinary_gradient}
\begin{align}
\frac{\partial L}{\partial\mu} &=
\frac{1}{\sigma\omega}(1 + \xi - \nu), \\
\frac{\partial L}{\partial\sigma} &=
\frac{1}{\sigma} \left[-1 + \frac{1}{\sigma \omega} (1 + \xi - \nu)(y - \mu) \right], \\
\frac{\partial L}{\partial\xi} &=
\frac{1}{\xi^2} \log \left( \omega (1 - \nu) \right) - \frac{1}{\sigma\omega} (1 + \xi - \nu)(y - \mu)\xi, \\
\omega &= 1 + \frac{\xi}{\sigma}(y - \mu),\\
\nu &= \omega^{-\frac{1}{\xi}}.
\end{align}
\end{subequations}
For gamma function $\Gamma(x) = \int_0^\infty \tau^{x-1} \exp (-\tau) d\tau$, digamma function $\psi(x) = \partial\log\Gamma(x)/ \partial x$, and Euler--Mascheroni constant $\gamma_\text{EM} = -\psi(1) \approx 0.5772$, Fisher information $\boldsymbol{I}(\boldsymbol{\theta})$ is given by \cite{prescott_maximum_1980}
\begin{subequations}\label{eq:fisher}
\begin{align}
I_{\mu\mu} &= \frac{1}{\sigma^2} p, \\
I_{\mu\sigma} &= -\frac{1}{\sigma^2\xi} \left( p - \Gamma(2+\xi) \right), \\
I_{\mu\xi} &= \frac{1}{\sigma\xi} \left( q - \frac{p}{\xi} \right), \\
I_{\sigma\sigma} &= \frac{1}{\sigma^2\xi^2} \left( 1 - 2\Gamma(2+\xi) + p \right), \\
I_{\sigma\xi} &= \frac{1}{\sigma\xi^2} \biggl(\frac{1}{\xi} (1 - \Gamma(2+\xi) + p) + 1 - \gamma_\text{EM} - q \biggr), \\
I_{\xi\xi} &= \frac{1}{\xi^2} \left(\frac{\pi^2}{6} +
\left(1 - \gamma_\text{EM} + \frac{1}{\xi}\right)^2 - \frac{2q}{\xi} + \frac{p}{\xi^2}\right), \\
p &= (1 + \xi)^2 \Gamma(1 + 2\xi), \\
q &= \Gamma(2 + \xi) \left(\psi(1 + \xi) + \frac{1 + \xi}{\xi}\right).
\end{align}
\end{subequations}
It has been discovered that the \gls{MLE} for \gls{GEV} does not exist for $\xi \leq -1$ and does not possess asymptotic consistency when $-1 < \xi \leq -0.5$ \cite{dey_extreme_2016}. On the other hand, it is well-known that \gls{NGD} on \gls{GEV} is not numerically well-behaved---in addition to the need to test for singularity $\xi=0$, where \autoref{eq:ordinary_gradient} and \autoref{eq:fisher} must be replaced with their corresponding limits, due to poles in $\Gamma(x), \psi(x)$, numerical evaluation of $\boldsymbol{I}(\hat{\boldsymbol{\theta}})$ often results in an ill-conditioned matrix that is extremely difficult to invert. As a result, checking and reconditioning of exceptional numerical cases are required to prevent divergence, resulting in computational inefficiency.
We instead adopt parameter estimation through the \gls{PWM} method \cite{hosking_estimation_1985}. Despite having a small bias, the \gls{PWM} is found by \cite{martins_generalized_2000} to have a lower bias than \gls{MLE} at small sample sizes, and the advantage can be crucial under limited data availability. In fact, due to its superior stability, \gls{PWM} is often preferred over \gls{MLE} in fields such as hydrology \cite{coles_introduction_2001}. We refer the reader to \cite{hosking_estimation_1985} for a comprehensive quantitative comparison between \gls{PWM} and \gls{MLE}. The \gls{PWM} estimate for \gls{GEV} is given by
\begin{subequations}\label{eq:pwm}
\begin{align}
\label{eq:pwm_mu}
\hat{\xi} &= -7.8590c - 2.9554c^2, \\
\label{eq:pwm_sigma}
\hat{\sigma} &=
\frac{(b_0 - 2b_1)\hat{\xi}}{\Gamma(1 - \hat{\xi}) (1 - 2^{\hat{\xi}})}, \\
\label{eq:pwm_xi}
\hat{\mu} &=
b_0 - \frac{\hat{\sigma}}{\hat{\xi}} (\Gamma(1 - \hat{\xi}) - 1),
\end{align}
\end{subequations}
and for $y'_1,\cdots, y'_N$ representing the block maxima sorted by value in ascending order, the quantities $b_0, b_1, b_2, c$ are given by
\begin{subequations}
\begin{align}
\label{eq:pwm_b0}
b_0 &= \frac{1}{N} \sum_{j=1}^{N} y'_j, \\
\label{eq:pwm_b1}
b_1 &= \frac{1}{N} \sum_{j=1}^{N} \frac{j-1}{N-1} y'_j, \\
\label{eq:pwm_b2}
b_2 &= \frac{1}{N} \sum_{j=1}^{N} \frac{(j-1)(j-2)}{(N-1)(N-2)} y'_j, \\
\label{eq:pwm_c}
c &= \frac{2b_1 - b_0}{3b_2 - b_0} - \frac{\log 2}{\log 3}.
\end{align}
\end{subequations}
\subsection{Construction of the conditional estimators}
\label{sec:conditional-estimators}
We base our procedure on \cite{breiman_classification_2017}, known as \gls{CART}. It is a recursive partitioning algorithm that, during fitting, dichotomizes the covariate-target pairs using a set of automatically discovered splitting rules, which are organized in the form of a tree. In the prediction phase, the splitting rules are applied to the given covariate in topological order, resulting in the activation of one of the leaf partitions whose estimate gives rise to the prediction.
For set of covariate dimensions $\mathcal{M}$, parent observation indices $\mathcal{P}$ and parent partition $(\boldsymbol{x}_{\mathcal{M},\mathcal{P}}, y_\mathcal{P})$, the child observation indices $L, \mathcal{R}$ resulting from splitting the parent in covariate dimension $m$ on threshold $u$ are:
\begin{subequations}\label{eq:children}
\begin{align}
\mathcal{L} &=\{p|p \in \mathcal{P} \land \boldsymbol{x}_{m,p} \leq u \} \label{eq:left_child} \\
\mathcal{R} &=\{p|p \in \mathcal{P} \land \boldsymbol{x}_{m,p} > u \} \label{eq:right_child}
\end{align}
\end{subequations}
The tuple $(m, u)$ is known as the splitting rule of the partition. In \gls{CART}, the selection of the optimal splitting rule is based on maximizing the reduction in squared error impurity, resulting in children in which target observations are similar in value. However, targets in the same partition do not necessarily have the same distribution. We hence modified \gls{CART}, using the \gls{GEV} log score \autoref{eq:log_score} to select the optimal splitting rule.
For covariate-target pairs $(\boldsymbol{x}_{\mathcal{M}, \mathcal{N}}, y_\mathcal{N})$, the procedure to construct the conditional estimators is described as follows:
\begin{enumerate}[S1]
\item\label{item:tree_fitting1} The modified \gls{CART} begins with the creation of a root node to which all covariate--target pairs are assigned.
\item\label{item:tree_fitting2}
A covariate dimension is selected, starting with $m=1$. The unique values in the dimension are sorted, considering the midpoints as candidates for the splitting threshold. For each candidate splitting rule $(m, u)$, transient partitions $y_L, y_\mathcal{R}$ are created via \autoref{eq:children}. The \gls{GEV} \gls{PWM} estimates and log scores are computed on $y_L, y_\mathcal{R}$ via \autoref{eq:pwm} and \autoref{eq:log_score}, respectively. For parent log score $L_p$, and children log scores $L_l, L_r$, we define the impurity drop $T$ as the reduction of the children log scores relative to the parent:
\begin{equation}
T = \frac{1}{L_p}(L_p - L_r - L_l)
\end{equation}
The splitting rule that results in the maximum impurity drop is selected as the optimal rule of the partition.
\item\label{item:tree_fitting3} The leaf nodes are scanned, and the leaf whose optimal rule results in the maximum impurity reduction is selected as the growing leaf. The optimal splitting rule of the leaf is finally applied to create the child partitions, which are inserted under the leaf. Step \ref{item:tree_fitting2} is repeated recursively on the new children.
\end{enumerate}
While \gls{CART} adopts post-processing procedures known as pruning to trim certain branches to reduce the complexity of the tree, we adopt early termination, also known as pre-pruning, to suppress over-fitted branches during tree construction. The conditions for early termination are:
\begin{itemize}
\item The optimal threshold search in Step \ref{item:tree_fitting2} is limited to those that result in partitions of at least a specified minimum size. A larger partition results in more accurate \gls{PWM} estimates. We found that a minimum partition size of 20--50 covariate--target pairs often yields satisfactory results.
\item No further children are allowed under a branch if the maximum impurity reduction of its leaf is lower than a specified limit $T_\text{crit}$. We found through experiments that $T_\text{crit}$ in the range of 0.0001--0.01 often results in satisfactory results.
\item The number of growing iterations is limited to a specified maximum. We found through experiments that it should be kept as large as computationally manageable, such as a value of $40$.
\end{itemize}
To illustrate the fitting procedure, we extracted the runtime data from one of the conditional estimators in the bagging ensemble during fitting of the example in \autoref{sec:example} and provided it in \autoref{fig:example_tree}. The artwork illustrates the top few nodes of only one of the conditional estimators, and the bagging ensemble consists of a large number of these conditional estimators. We encourage the reader to cross-examine \autoref{fig:example_tree} with the procedure described in Steps \ref{item:tree_fitting1} to \ref{item:tree_fitting3} to understand the fitting process.
\begin{figure}
\centering
\includegraphics[width=0.9\linewidth]{999-fig-004.pdf}
\caption{Structure of one of the conditional estimators created during fitting. The artwork is intended to illustrate the fitting procedure. Deeper nodes are not shown to conserve ink.}
\label{fig:example_tree}
\end{figure}
\section{Application-neutral example}
\label{sec:example}
We envision that our approach may become broadly applicable in the scientific domain and herein present an application-neutral example to benefit a wider audience before presenting a power system-specific case study in the next section. Different from the case study, the theoretical nature of this example enables the calculation of a true reference for the quantile estimate and allows straightforward comparisons of our approach and a competitor against the known reference.
The data for the example consists of $\mathcal{N}=\{1,\cdots, 1000\}$ covariate--target pairs where the covariate $\boldsymbol{x}_{\mathcal{M},\mathcal{N}}$ has one dimension $\mathcal{M}=\{1\}$ and the pairs are synthesized according to the following rules:
\begin{subequations}
\begin{align}
x_{1,\mathcal{N}} &\in [0, \pi], \\
y_\mathcal{N} &\sim \text{GEV}(\mu_\mathcal{N}, \sigma_\mathcal{N}, \xi_\mathcal{N} | \boldsymbol{x}_{\mathcal{M},\mathcal{N}}),
\end{align}
\end{subequations}
where the instantaneous distribution parameters are given by
\begin{subequations}\label{eq:synthetic_example}
\begin{align}
\mu_n(x_{1,n}) &= \cos(1.23 x_{1,n}) + 0.3 \cos(4.56 x_{1,n}),\\
\xi_n(x_{1,n}) &= 0.3 \cos(5.67 x_{1,n}), \\
\sigma_n(x_{1,n}) &= 1 + 0.1 \cos(6.78 x_{1,n}).
\end{align}
\end{subequations}
The proposed log-score-based splitting criteria are sensitive to variations in the data, and thus, the ensemble approach we adopted is crucial in improving the quality of the estimates. Using number of ensemble members $K=50$, minimum partition size of $20$, and minimum impurity reduction $T_\text{crit}=0.0001$, resample ratio $\rho=1$, the \gls{GEV} distribution estimated using our approach is visualized in the form of the $90\%$ prediction interval against the synthesized block maxima in \autoref{fig:synthetic_observations}, and the parameter estimates are shown in \autoref{fig:synthetic_params}.
\begin{figure}[htb!]
\centering
\includegraphics[width=\textwidth]{999-fig-005.pdf}
\caption{The synthesized block maxima and the nonstationary GEV estimated using our approach visualized in the form of its expectation and the $90\%$ prediction interval (shaded region). GEV---generalized extreme value distribution.}
\label{fig:synthetic_observations}
\end{figure}
A robust estimator should produce consistent estimates on samples drawn from the same distribution, despite the observations being different each time. The bias relates to the asymptotic accuracy of the estimator, and the variance relates to the stability of the estimate. The achievable minimum asymptotic variance of the estimates produced by an unbiased estimator known as the \gls{MVUE} is given by the \gls{CRB} as follows:
\begin{equation}
\text{Var}(\hat{\boldsymbol{\theta}}) \geq I^{-1}(\boldsymbol{\theta}),
\label{eq:crb}
\end{equation}
where $I(\boldsymbol{\theta})$ is the \gls{GEV} Fisher information given by \autoref{eq:fisher}. An unbiased estimator that achieves the \gls{CRB} is considered statistically most efficient. To contrast our approach, we graphically represent the $90\%$ parameter confidence intervals of the \gls{MVUE} calculated using the diagonal elements of \autoref{eq:crb} in the shaded regions of \autoref{fig:synthetic_params}. Since the \gls{PWM} method is biased, a test for significant parameter deviation, such as the Wald test \cite{wald_tests_1943}, would not be well-calibrated. Nonetheless, it can be observed that our estimates fall within the interval, a graphical indication that such tests would likely be unable to reject the hypothesis that there are no statistically significant deviations between our estimates and the true parameters.
\begin{figure}[htb!]
\centering
\includegraphics[width=\textwidth]{999-fig-006.pdf}
\caption{The true parameters of the distribution that synthesized the block maxima, the $90\%$ confidence interval of a theoretical and ideal MVUE (shaded region), and the estimates produced by our approach. MVUE---minimum variance unbiased estimator.}
\label{fig:synthetic_params}
\end{figure}
\subsection{Comparison with a state-of-the-art parametric competitor}
\Gls{GAMLSS} \cite{rigby_generalized_2005} provides a flexible framework that can approximate the parameters of a variety of distribution families as smooth functions of the covariate. It is widely used in applications such as climatology \cite{dabernig_simultaneous_2017} and public health \cite{giacomet_distributional_2023} to model the variation of probability densities in response to environmental factors. However, GAMLSS comes with several notable practical drawbacks:
\begin{itemize}
\item \Gls{GAMLSS} requires the labor-intensive manual specification of formulas for all three \gls{GEV} parameters through trial-and-error, which makes the forecasting model prone to misspecification.
\item Spline-based function approximations are not well-suited to capturing discontinuous or highly nonlinear relationships that may be encountered during peak demand probabilistic modeling.
\item \Gls{GAMLSS} can suffer from over-smoothing or under-fitting, where the estimates fail to adequately capture the behavior of the distribution parameters.
\end{itemize}
To empirically validate these limitations, we compared our method with \gls{GAMLSS} on the same dataset described earlier. All three \gls{GEV} parameters in the \gls{GAMLSS} model are approximated using penalized B-splines of the covariate $\boldsymbol{x}_{1,n}$. The estimates of our method and those of \gls{GAMLSS} are displayed in \autoref{fig:synthetic_params_gamlss} against the true parameters.
\begin{figure}[htb!]
\centering
\includegraphics[width=\textwidth]{999-fig-007.pdf}
\caption{The true parameters of the distribution that synthesized the block maxima, the estimates produced by our approach, and the estimates produced by GAMLSS. GAMLSS---generalized additive model for location, scale, and shape.}
\label{fig:synthetic_params_gamlss}
\end{figure}
For a more precise quantitative comparison, we use the \gls{CRPS}, a robust scoring metric frequently used in forecast verification in numerical weather prediction. The \gls{CRPS} compares the \gls{CDF} of the predicted density against an ideal \gls{CDF} corresponding to each observation. For gamma function $\Gamma(x) = \int_0^\infty \tau^{x-1} \exp (-\tau) d\tau$, lower incomplete gamma function $\Gamma_l(s,x)=\int_0^x t^{s-1} \exp(-t) dt$, exponential integral function $\text{Ei}(x)=\int_{-\infty}^x e^t/t dt$, and Euler--Mascheroni constant $\gamma_\text{EM} = -\psi(1) \approx 0.5772$, the \gls{CRPS} formula for \gls{GEV} is derived by \cite{friederichs_forecast_2012} as follows:
\begin{equation}
\text{S}(y, \hat{\boldsymbol{\theta}}) =
\begin{cases}
\begin{aligned}
&\left(\hat{\mu} - y - \frac{\hat{\sigma}}{\hat{\xi}} \right)
\left(1 - 2 F_{\xi\neq 0}(y) \right) \\
&\quad -\frac{\hat\sigma}{\hat\xi} \left(
2^{\hat\xi} \Gamma(1 - \hat\xi) - 2 \Gamma_l (1 - \hat\xi, -\log F_{\xi\neq 0}(y)) \right),
\end{aligned} & \hat\xi \neq 0, \\
\hat{\mu} - y + \hat{\sigma} (\gamma_\text{EM} - \log 2) - 2\hat{\sigma} \text{Ei} (\log F_{\xi=0}(y)), & \hat\xi = 0,
\end{cases}
\label{eq:crps_gev}
\end{equation}
where $F(y)$ is the \gls{CDF} of \gls{GEV}, given by
\begin{equation}
F(y) = \exp(-\tau),
\end{equation}
and $\tau$ is shown in \autoref{eq:tau}. The \gls{CRPS} of our method and \gls{GAMLSS} on the previously described dataset is listed in \autoref{tab:ours_and_gamlss_crps}. The scores shown are the average \gls{CRPS} over all observations, and the per-observation \gls{CRPS} is calculated via \autoref{eq:crps_gev}. A lower score indicates better agreement between the observations and the \gls{CDF} of the estimated density.
\begin{table}[htb!]
\centering
\begin{tabular}{cc}
\hline
GAMLSS CRPS & Our CRPS \\
\hline
0.719 & \textbf{0.677} \\
\hline
\end{tabular}
\caption{Comparison of the CRPS between our approach and GAMLSS in the application-neutral example. CRPS---continuous ranked probability score, GAMLSS---generalized additive model for location, scale, and shape.}
\label{tab:ours_and_gamlss_crps}
\end{table}
\subsection{Exploring the limitation of a quantile-based non-parametric competitor}
In terms of nonparametric competitors, since the block maxima in the example follow a nonlinear relationship with the covariate $\boldsymbol{x}_{\mathcal{M},\mathcal{N}}$, linear approaches such as \gls{QR} would discernibly be out-competed by our approach, which can model nonlinear relationships.
To present a fair case-by-case comparison across different quantiles, we computed the block maxima quantile estimates from our results using the inverse \gls{CDF} shown in \autoref{eq:inverse_cdf} and compared them with the true quantile targets based on \autoref{eq:synthetic_example} and quantile estimates of \gls{QRF} \cite{johnson_quantile-forest_2024}. The results are shown in \autoref{fig:synthetic_example_quantile_losses} and \autoref{fig:synthetic_black_swan}.
\begin{equation}
F^{-1}(\alpha; \boldsymbol{\theta})=
\begin{cases}
\mu - \sigma \log(-\log \alpha) & \xi = 0, \\
\mu + \frac{\sigma}{\xi} \left((-\log \alpha)^{-\xi} -1 \right) & \xi \neq 0.
\end{cases}
\label{eq:inverse_cdf}
\end{equation}
It can be observed in \autoref{fig:synthetic_example_quantile_losses} that our approach outperforms \gls{QRF} in all evaluated quantiles and especially at the \gls{LPHC} quantiles. In \autoref{fig:synthetic_black_swan}, for quantiles $0.1, 0.5$, and $0.9$, the estimates of \gls{QRF} are especially noisy compared to our estimates and the true block maxima targets, suggesting some level of over-fitting by \gls{QRF}. Given that an over-fitted \gls{QRF} possesses increased sensitivity to extreme observations, for \gls{LPHC} quantiles $0.999$ and $0.999999$, however, the \gls{QRF} estimates deviate significantly from the true targets, while the quantile estimates produced by our approach are in significantly better agreement with the true targets. The scarcity of extreme observations limits the ability of quantile-based methods, including highly sophisticated deep-learning-based quantile regression methods, to accurately approximate distribution tails. Quantile methods are often out-competed in terms of accuracy and trustworthiness by \gls{EVA} in critical applications such as finance and insurance underwriting. For more comprehensive comparisons between quantile-based methods and \gls{EVA}, we refer the reader to \cite{mcneil_quantitative_2015, velthoen_gradient_2022, gnecco_extremal_2024}.
\begin{figure}[htb!]
\centering
\begin{minipage}[t]{0.48\textwidth}
\centering
\includegraphics[width=\textwidth, page=1]{999-fig-008.pdf}
\end{minipage}
\begin{minipage}[t]{0.48\textwidth}
\centering
\includegraphics[width=\textwidth, page=2]{999-fig-008.pdf}
\end{minipage}
\caption{The errors between quantile estimates given by QRF and our approach against the true quantile target of the distribution that synthesized the block maxima. QRF---quantile regression forest.}
\label{fig:synthetic_example_quantile_losses}
\end{figure}
\begin{figure}[htb!]
\centering
\begin{minipage}[t]{\textwidth}
\centering
\includegraphics[width=\textwidth, page=1]{999-fig-009.pdf}
\end{minipage}
\begin{minipage}[t]{\textwidth}
\centering
\includegraphics[width=\textwidth, page=2]{999-fig-009.pdf}
\end{minipage}
\begin{minipage}[t]{\textwidth}
\centering
\includegraphics[width=\textwidth, page=3]{999-fig-009.pdf}
\end{minipage}
\begin{minipage}[t]{\textwidth}
\centering
\includegraphics[width=\textwidth, page=4]{999-fig-009.pdf}
\end{minipage}
\caption{The quantile estimates given by QRF and our approach and the true quantile target of the distribution that synthesized the block maxima. Note that the vertical axis differs in scale considerably between plots. QRF---quantile regression forest.}
\label{fig:synthetic_black_swan}
\end{figure}
\section{Case study---day-ahead scheduling capacity determination}
\label{sec:case_study}
The PJM interconnection is a \gls{RTO} serving 65 million people across 13 states and the District of Columbia. The system demand ranges between $\SI{80}{\giga\watt}$ to $\SI{150}{\giga\watt}$, the summer peak in 2024 exceeding $\SI{152}{\giga\watt}$. The simplified block diagram of the scheduling workflow of PJM is shown in \autoref{fig:pjm_market}.
\begin{figure}[htb!]
\centering
\includegraphics[width=0.7\textwidth]{999-fig-010.pdf}
\caption{Simplified block diagram of the scheduling workflow of PJM. DAEM---day-ahead energy market, DAASM---day-ahead ancillary service market, SCED---security-constrained economic dispatch, RAC---reliability assessment and commitment, RTEM---real-time energy market, RTASM---real-time ancillary service market.}
\label{fig:pjm_market}
\end{figure}
According to PJM Manual 11 \cite{ciabattoni_pjm_2024}, offers and bids are submitted into the day-ahead markets by market participants. Following the closure of the markets at 11:00 ahead of the operating day, PJM performs market clearing through the \gls{SCED} engine to select the most economic generation offers while ensuring generation is sufficient to satisfy demand. The hourly generation schedule, \gls{LMP}, and \gls{MCP} determined by \gls{SCED} are posted before 13:30, and winners are allowed to make adjustments to the cleared schedules through the rebidding market, which closes at 14:45. PJM then performs a second resource commitment known as the \gls{RAC}, which ensures sufficient generation capacity is scheduled to meet the reserve requirements for the next operating day as the activities progress to the real-time markets at midnight.
\begin{figure}[htb!]
\centering
\includegraphics[width=0.55\textwidth]{999-fig-011.pdf}
\caption{Composition of the day-ahead scheduling capacity.}
\label{fig:pjm_capacity}
\end{figure}
To prepare for possible contingencies and maintain energy balance under operational uncertainty, additional generation resources known as reserves are scheduled by \gls{RAC} based on \autoref{fig:pjm_capacity} and placed on standby during the operating day. In this case study, we apply our approach and demonstrate a risk-based \gls{DASC} requirement determination method. For simplicity, the case study only considers total system demand within the PJM \gls{RTO} region. External bilateral sales and scheduled interchanges with neighboring balancing authority areas are ignored.
\subsection{Data origin and fitting}
The PJM Data Miner 2 platform \cite{noauthor_data_nodate} provides a variety of publicly-accessible datasets.
We obtained the demand \gls{HIA}, hourly demand forecasts, and historical day-ahead temperature forecasts for study years 2022--2024 from the platform.
The PJM demand forecasts are updated every 6 hours and predict the demand for each hour of an entire day in the form of single-valued point estimates.
The forecasts are highly accurate and according to PJM Manual 11 \cite{ciabattoni_pjm_2024}, are obtained using a combination of neural net and pattern-matching approaches.
Based on our recommendation in \autoref{sec:autocorrelation}, we create daily peak probabilistic predictions based on the PJM point prediction and delegate the responsibility of modeling autocorrelation to the PJM forecast---we assume the daily peak follows the conditional distribution $y\sim \text{GEV}(\boldsymbol{\theta}|\boldsymbol{x})$ whose covariate $\boldsymbol{x}$ is shown in \autoref{fig:pjm_training}. To integrate our approach into the PJM workflow in \autoref{fig:pjm_market}, the capacity requirements need to be available at least 36 hours in advance for the \gls{RAC} run at 14:45, leading us to adopt the demand forecasts produced at 05:45 ahead of the operating day.
\begin{figure}[htb!]
\centering
\includegraphics[width=0.7\textwidth, page=1]{999-fig-012.pdf}
\caption{The fitting data contains only the daily peaks and the corresponding covariate, whose structure is shown in the figure. DA---day-ahead.}
\label{fig:pjm_training}
\end{figure}
The forecasting ensemble is fitted to the covariate-peak pairs $(\boldsymbol{x}_{\mathcal{M}, \mathcal{N}}, y_\mathcal{N})$ in study years 2022--2023 using number of ensemble members $K=50$, resampling ratio $\rho=1$, the minimum impurity drop $T_\text{crit}=0.05$, and the minimum number of covariate-target pairs of each partition limited to no less than $30$. Data for test year 2024 is held confidential to the fitting procedures.
We define the residual $r_\mathcal{N}$ as the difference between the statistical expectation of the predicted peak distribution and the actual daily peak $y_\mathcal{N}$:
\begin{equation}
r_\mathcal{N} \triangleq \hat{y}_\mathcal{N} - y_\mathcal{N}
=
\mathbb{E}\left[\text{GEV}(\hat{\boldsymbol{\theta}}|\boldsymbol{x}_{\mathcal{M}, \mathcal{N}})\right] - y_\mathcal{N},
\end{equation}
and the expectation is given by
\begin{equation}
\mathbb{E}\left[\text{GEV}(\boldsymbol{\theta})\right] =
\begin{cases}
\mu + \frac{\sigma}{\xi} \left( \Gamma(1-\xi) - 1\right) & \xi \in (-\infty, 1)\setminus \{0\},\\
\mu + \sigma \gamma_\text{EM} & \xi = 0, \\
\infty & \xi \geq 1.
\end{cases}
\end{equation}
The residuals of a robust forecasting algorithm should follow a stationary normal distribution with a small mean and variance. It can be observed from \autoref{fig:pjm_residual} that the residuals in study years 2022--2023 appear to follow a normal distribution.
Following fitting, covariates of test year 2024 are supplied to the ensemble, resulting in forecasts for the entire study year in the form of \gls{GEV} parameter estimates for the daily peak that are updated hourly as conditions change.
As a consequence of including the highly accurate PJM point forecast in the covariate, the \gls{MAE} of the forecasted daily peak expectation against the actual daily peaks in test year 2024 is $\SI{2814.28}{\mega\watt}$, or $1.84\%$ of the $\SI{153}{\giga\watt}$ total system demand. The mean under-forecasted forecasting error of the daily peak is $\SI{1850.15}{\mega\watt}$, or $1.21\%$ of the total system demand.
\begin{figure}[ht!]
\centering
\begin{minipage}[t]{0.49\textwidth}
\centering
\includegraphics[width=\textwidth, page=1]{999-fig-013.pdf}
\end{minipage}
\hfill
\begin{minipage}[t]{0.47\textwidth}
\centering
\includegraphics[width=\textwidth, page=2]{999-fig-013.pdf}
\end{minipage}
\caption{Quantile-quantile plot and histogram of the daily peak residuals in study years 2022--2023.}
\label{fig:pjm_residual}
\end{figure}
\subsection{Results: day-ahead scheduling capacity}
According to PJM Manual 13 \cite{hatch_pjm_2024}, the \gls{DASC} 30-minute reserve requirement is calculated based on the annual peak load forecast adjusted for the under-forecasted load-forecasting error and generator \gls{FOR}. On the other hand, the determination of required reserves in long- and near-term planning follows the ``one day in ten years" guideline specified by \gls{NERC} standard BAL-502-RF-03 \cite{north_american_electric_reliability_corporation_bal-502-rf-03_2017} based on the \gls{LOLP}, defined as the probability of demand exceeding the available generation capacity. The \gls{NERC} guideline requires the annual \gls{LOLP} to be equal to 0.1 and generalizes into daily \gls{LOLP} risk requirement $\eta$ as follows:
\begin{equation}
\eta = \frac{\SI{0.1}{\si{Events}\per\si{year}}}{\SI{365}{\si{days}\per\si{year}}}
= \SI{273.9e-6}{\si{Events}\per\day}
\label{eq:nerc_risk}
\end{equation}
It can be argued that it is possible for the PJM method to over-commit the \gls{DASC} over low-risk periods of the year, resulting in unnecessary increases in wholesale energy prices. Numerous papers propose alternatives and demonstrate risk-averse stochastic scheduling methods, such as \cite{farzan_toward_2015,bahramara_day-ahead_2022} on microgrids, and \cite{guo_risk-averse_2022} on renewable energy systems. The required generation capacity to control the \gls{LOLP} to risk level $\eta$ corresponds to an extreme quantile and as demonstrated in \autoref{sec:example}, is difficult to accurately estimate using non-\gls{EVA} approaches. Therefore, we critique the aforementioned papers for the lack of a comprehensive statistical framework to accurately model extreme values and critical system risks.
For the joint probability $\Pr(y,\boldsymbol{x}_n)$ of daily peak of severity $y$ occurring under hourly condition $\boldsymbol{x}_n$, the \gls{DASC} that can accommodate the daily peak under risk level $\eta$ satisfies
\begin{equation}
\begin{aligned}
\Pr(y > \text{DASC}_n, \boldsymbol{x}_n) &\leq \eta, \\
\Rightarrow
\Pr(y > \text{DASC}_n;\hat{\boldsymbol{\theta}}_n|\boldsymbol{x}_n) \Pr(\boldsymbol{x}_n) &\leq \eta,
\end{aligned}
\end{equation}
where the conditional daily peak density $\Pr(y;\hat{\boldsymbol{\theta}}_n|\boldsymbol{x}_n)$ is estimated using our approach. As discussed in \autoref{sec:problem_statement}, under worst-case baseline probability $\Pr(\boldsymbol{x}_n) \equiv 1$,
\begin{equation}
\begin{aligned}
\Pr(y > \text{DASC}_n;\hat{\boldsymbol{\theta}}_n|\boldsymbol{x}_n) &\leq \eta, \\
\Rightarrow \text{DASC}_n &\geq F^{-1} (\alpha;\hat{\boldsymbol{\theta}}_n) = \text{VaR}_n(\alpha),
\end{aligned}\label{eq:dasc}
\end{equation}
where confidence $\alpha=1 - \eta$ and $F^{-1}(\alpha;\boldsymbol{\theta}_n)$, also known as the \gls{VaR}, is calculated via \autoref{eq:inverse_cdf}. The relationship between the \gls{VaR}, the inverse \gls{CDF}, the \gls{DASC}, and the \gls{LOLP} is displayed in \autoref{fig:quantile_function}.
\begin{figure}[htb!]
\centering
\includegraphics[width=0.7\textwidth]{999-fig-014.pdf}
\caption{Relationship between the inverse CDF, the VaR, the DASC, and the LOLP. CDF---cumulative distribution function, VaR---value-at-risk, DASC---day-ahead scheduling capacity, LOLP---loss-of-load probability.}
\label{fig:quantile_function}
\end{figure}
The \gls{DASC} determined from the daily risk requirement $\eta$ changes each hour as conditions change and is displayed in \autoref{fig:pjm_testing_prediction}, contrasted by the actual demand \gls{HIA} and the PJM \gls{DASC} determined from the economic maximum of generating offers cleared in the \gls{DAEM}. Over the two high-risk months shown in \autoref{fig:pjm_testing_prediction}, our \gls{DASC} enveloped the demand curve and it can be observed that around the $\SI{133}{\giga\watt}$ winter peak on January 17 and around the $\SI{152}{\giga\watt}$ summer peak on July 16, compared to previous and subsequent days, our approach increased the capacity to compensate for the increased risk, while the PJM \gls{DASC} stayed largely unchanged, indicating a potential for over-commitment.
The sum of the committed hourly capacity of PJM in study year 2024 is $\SI{1470573}{\unit{GWh}}$, while that of approach is $\SI{910052}{\unit{GWh}}$, a $38\%$ reduction.
It can be concluded that adopting our approach would result in significant cost savings by committing capacity only as much as needed to maintain the risk requirement.
\begin{figure}[htb!]
\centering
\begin{minipage}[t]{\textwidth}
\centering
\includegraphics[width=\textwidth, page=1]{999-fig-015.pdf}
\end{minipage}
\hfill
\begin{minipage}[t]{\textwidth}
\centering
\includegraphics[width=\textwidth, page=2]{999-fig-015.pdf}
\end{minipage}
\caption{Comparison between the DASC determined from our approach, the DASC determined by PJM, and the demand HIA in high-risk months of study year 2024. DASC---day-ahead scheduling capacity, HIA---hourly integrated average.}
\label{fig:pjm_testing_prediction}
\end{figure}
\subsection{Results: expected unserved energy}
The \gls{EUE} is defined as the expected total amount of unserved energy in a time horizon due to capacity shortage. It is typically presented in megawatt hours and used as an annual risk indicator. Using our approach, however, we can estimate the \gls{EUE} in daily resolution. In a maximum capacity emergency, the day-ahead \gls{EUE} can help the \gls{RTO} identify the most likely amount and duration of load shedding during the operating day to restore energy balance, whether energy conservation directives and \gls{DSM} alone would suffice, and whether critical loads would be affected. Assuming worst-case baseline probability $\Pr(\boldsymbol{x}_n)\equiv1$ where condition $\boldsymbol{x}_n$ of hour $n$ always results in the daily peak, the daily \gls{EUE} under the \gls{DASC} determined from \autoref{eq:dasc} based on confidence $\alpha=1 - \eta$ is given by
\begin{equation}
\begin{aligned}
\text{EUE}
&= \sum_{n=1}^{24} \left(
\int_{\text{VaR}_n(\alpha)}^\infty
\left(y - \text{VaR}_n(\alpha)\right)
\Pr(y,\boldsymbol{x}_n) dy \right) \\
&= \sum_{n=1}^{24} \int_{\text{VaR}_n(\alpha)}^\infty dy
\left(
y\Pr(y;\hat{\boldsymbol{\theta}}|\boldsymbol{x}_n)
- \text{VaR}_n(\alpha)
\Pr(y;\hat{\boldsymbol{\theta}}|\boldsymbol{x}_n)
\right)\\
&= (1 - \alpha) \sum_{n=1}^{24} \left(
\text{CVaR}_n(\alpha)
- \text{VaR}_n(\alpha)
\right),
\end{aligned}\label{eq:eue}
\end{equation}
and for lower incomplete gamma function $\Gamma_l(s,x)=\int_0^x t^{s-1} \exp(-t) dt$, logarithmic integral function $\text{li}(x)=\int_0^x dx/\log x$, the \gls{CVaR} is given by
\begin{equation}
\text{CVaR}(\alpha) =
\begin{cases}
\mu + \frac{\sigma}{1 - \alpha}
\left[
\gamma_\text{EM} -
\text{li}(\alpha) +
\alpha\log(-\log\alpha)
\right] & \xi = 0,\\
\mu + \frac{\sigma}{(1-\alpha)\xi} \left[
\Gamma_l(1 - \xi, -\log\alpha) - (1-\alpha)
\right] & \xi \neq 0.
\end{cases}
\end{equation}
\begin{figure}[htb!]
\centering
\includegraphics[width=\textwidth]{999-fig-016.pdf}
\caption{Predicted daily EUE in study year 2024. EUE---expected unserved energy.}
\label{fig:pjm_eue}
\end{figure}
The \gls{EUE} determined from \autoref{eq:eue} for study year 2024 is shown in \autoref{fig:pjm_eue}. It can be observed that in terms of magnitude, severe \gls{EUE} occurs in the winter, while in terms of surface area, however, the \gls{EUE} accumulates in the summer. We calculated the annual sum of the daily \gls{EUE} to compare with annual \gls{EUE} metrics from the industry. Assuming the adoption of our approach for the \gls{DASC} via the \gls{LOLP} guideline in \autoref{eq:nerc_risk}, the estimated annual \gls{EUE} in study year 2024 is $\SI{32.37}{\unit{MWh}}$, or $\SI{0.04}{\unit{PPM}}$ (parts per million) of the $\SI{799269.28}{\unit{GWh}}$ total demand. The \gls{EUE} of PJM is not publicly disclosed. However, an industry presentation reported a normalized \gls{EUE} of $\SI{1}{\unit{PPM}}$ \cite{ming_role_2024} for a different \gls{RTO}.
\subsection{Results: exploring a state-of-the-art competitor}
We attempted to apply the previously described competitor framework \gls{GAMLSS}\footnote{\texttt{gamlss} R package, version 5.4-22} to the case study using the same covariate structure shown in \ref{fig:pjm_training}. The location, scale, and shape parameters of the GEV are modeled as penalized B-splines of the PJM day-ahead demand forecast and PJM day-ahead ambient temperature. Day of week, day of month, and month are treated as categorical variables. Despite attempting different covariate element combinations, the competitor \gls{GAMLSS} failed to converge in all tested configurations. This reinforces the limitation of \gls{GAMLSS} in irregular data settings described in previous sections and is consistent with the findings of \cite{mayr_gamlss_2010}. Consequently, \gls{GAMLSS} is excluded from comparison in this case study. We refer the reader to \autoref{sec:example} where \gls{GAMLSS} successfully converged for comparison of \gls{GAMLSS} with our approach.
\section{Conclusion}
Risk-sensitive power system planning and scheduling require accurately characterizing the probability and severity of the peak demand.
To overcome the stationarity limitation of classical \gls{EVA}, energy regulatory and oversight agencies combine \gls{EVA} with Mont\'e-Carlo simulations for annual peak assessments, but per-scenario simulations become intractable at higher time resolutions, such as day-ahead scheduling.
Whereas literature favors assumption-free quantile forecasting methods that can model nonstationarity, we demonstrated through an example that such non-\gls{EVA} approaches fail to estimate critical “black swan” events due to the rarity of observations at extreme quantiles. In our case study, we reinforced findings in the literature that the state-of-the-art alternative \gls{GAMLSS} can suffer from convergence issues in high-dimensional covariate cases.
To adopt EVA in higher time resolutions, an innovative nonstationary estimator must be developed.
Leveraging established \gls{ML} techniques and \gls{EVA}, we devised a rigorous conditional statistical framework to model nonstationary extrema densities, clearly presented the assumptions, and addressed the weakening of the framework due to autocorrelation. Based on the framework, we developed and elaborated on a novel nonstationary \gls{EVA} parameter estimation approach, contrasted it with a theoretically most efficient estimator, state-of-the-art \gls{GAMLSS}, and a quantile-based competitor through an experiment, and presented its real-world application through a case study. As we discovered from the case study, the capacity determination method in the industry lacks resolution and cannot distinguish high and low-risk periods of the year, resulting in over-commitment of resources. Our approach only schedules as much capacity as needed to maintain the given risk requirement, achieving a $38\%$ reduction in yearly total committed capacity, resulting in potentially significant savings in wholesale electricity prices. This paper opens the door to numerous research directions utilizing nonstationary \gls{EVA} within and beyond power systems engineering.
\section*{Acknowledgements}
\begin{itemize}
\item This paper is partly derived from the work supported by the U.S. Department of Energy’s Office of Energy Efficiency and Renewable Energy (EERE) through the Solar Energy Technology Office (SETO) under Award DE-EE0009357. We thank the agency for its financial support. The funding source has no involvement in
the development of the methodology, design, preparation, or interpretation of the case study, or the decision to submit for publication.
\item We would like to thank Haley Northrup for her support in improving this paper's organization, wording, clarity, and impact-related aspects.
\end{itemize}
\section*{Declaration of generative AI and AI-assisted technologies in the writing process}
During the preparation of this work, the authors utilized generative \gls{AI} tools that include ChatGPT, ScholarGPT, and Consensus to assist in the discovery of relevant publications in academia and from the industry, and NotebookLM in the indexing and categorization of acquired public-domain training materials, presentations, manuals, and white papers obtained from PJM, ISO New England, and \gls{NERC}. Although some statements are informed by \gls{AI}-generated text, the paper does not contain \gls{AI}-generated content, and information provided by \gls{AI} has been critically examined, thoroughly substantiated, and rigorously vetted by the authors. The authors take full responsibility for the content of the published article.
\begin{thebibliography}{10}
\expandafter\ifx\csname url\endcsname\relax
\def\url#1{\texttt{#1}}\fi
\expandafter\ifx\csname urlprefix\endcsname\relax\def\urlprefix{URL }\fi
\expandafter\ifx\csname href\endcsname\relax
\def\href#1#2{#2} \def\path#1{#1}\fi
\bibitem{flores_2021_2023}
N.~M. Flores, H.~McBrien, V.~Do, M.~V. Kiang, J.~Schlegelmilch, J.~A. Casey, The 2021 {Texas} {Power} {Crisis}: distribution, duration, and disparities, Journal of Exposure Science \& Environmental Epidemiology 33~(1) (2023) 21--31.
\newblock \href {https://doi.org/10.1038/s41370-022-00462-5} {\path{doi:10.1038/s41370-022-00462-5}}.
\bibitem{nerc_probabilistic_2016}
{NERC}, \href{https://www.nerc.com/comm/RSTC/PAWG/proba_technical_guideline_document_08082014.pdf}{Probabilistic {Assessment}}, Technical {Guideline} {Document}, North American Electric Reliability Cooperation (Aug. 2016).
\newline\urlprefix\url{https://www.nerc.com/comm/RSTC/PAWG/proba_technical_guideline_document_08082014.pdf}
\bibitem{george_stephen_preliminary_2023}
S.~George, J.~Zhao, \href{https://www.iso-ne.com/static-assets/documents/2023/05/a10_operational_impact_of_extreme_weather_events.pdf}{Preliminary {Results} of {Energy} {Adequacy} {Studies} for {Winter} 2027} (May 2023).
\newline\urlprefix\url{https://www.iso-ne.com/static-assets/documents/2023/05/a10_operational_impact_of_extreme_weather_events.pdf}
\bibitem{dillon_integer_1978}
T.~S. Dillon, K.~W. Edwin, H.-D. Kochs, R.~J. Taud, Integer {Programming} {Approach} to the {Problem} of {Optimal} {Unit} {Commitment} with {Probabilistic} {Reserve} {Determination}, IEEE Transactions on Power Apparatus and Systems PAS-97~(6) (1978) 2154--2166.
\newblock \href {https://doi.org/10.1109/TPAS.1978.354719} {\path{doi:10.1109/TPAS.1978.354719}}.
\bibitem{shi_ant_2004}
L.~Shi, J.~Hao, J.~Zhou, G.~Xu, Ant colony optimization algorithm with random perturbation behavior to the problem of optimal unit commitment with probabilistic spinning reserve determination, Electric Power Systems Research 69~(2) (2004) 295--303.
\newblock \href {https://doi.org/10.1016/j.epsr.2003.10.008} {\path{doi:10.1016/j.epsr.2003.10.008}}.
\bibitem{haupt_use_2019}
S.~E. Haupt, M.~Garcia~Casado, M.~Davidson, J.~Dobschinski, P.~Du, M.~Lange, T.~Miller, C.~Mohrlen, A.~Motley, R.~Pestana, J.~Zack, The {Use} of {Probabilistic} {Forecasts}: {Applying} {Them} in {Theory} and {Practice}, IEEE Power and Energy Magazine 17~(6) (2019) 46--57.
\newblock \href {https://doi.org/10.1109/MPE.2019.2932639} {\path{doi:10.1109/MPE.2019.2932639}}.
\bibitem{impram_challenges_2020}
S.~Impram, S.~Varbak~Nese, B.~Oral, Challenges of renewable energy penetration on power system flexibility: {A} survey, Energy Strategy Reviews 31 (2020) 100539.
\newblock \href {https://doi.org/10.1016/j.esr.2020.100539} {\path{doi:10.1016/j.esr.2020.100539}}.
\bibitem{sweeney_future_2020}
C.~Sweeney, R.~J. Bessa, J.~Browell, P.~Pinson, The future of forecasting for renewable energy, WIREs Energy and Environment 9~(2) (2020) e365.
\newblock \href {https://doi.org/10.1002/wene.365} {\path{doi:10.1002/wene.365}}.
\bibitem{north_american_electric_reliability_corporation_bal-502-rf-03_2017}
{North American Electric Reliability Corporation}, {BAL}-502-{RF}-03: {Planning} {Resource} {Adequacy} {Analysis}, {Assessment} and {Documentation} (Oct. 2017).
\bibitem{daniel_cornelis_van_holst_pellekaan_draft_nodate}
{Daniel Cornelis van Holst Pellekaan}, \href{https://web.archive.org.au/awa/20200921005256mp_/http:/www.coagenergycouncil.gov.au/sites/prod.energycouncil/files/publications/documents/ESB%20Consultation%20-%20Draft%20National%20Electricity%20Amendment%20%28Interim%20Reliability%20Measure%29%20Rule%202020.pdf}{Draft {National} {Electricity} {Amendment} ({Interim} {Reliability} {Measure}) {Rule} 2020}.
\newline\urlprefix\url{https://web.archive.org.au/awa/20200921005256mp_/http:/www.coagenergycouncil.gov.au/sites/prod.energycouncil/files/publications/documents/ESB%20Consultation%20-%20Draft%20National%20Electricity%20Amendment%20%28Interim%20Reliability%20Measure%29%20Rule%202020.pdf}
\bibitem{noauthor_regulation_2019}
Regulation ({EU}) 2019/941 of the {European} {Parliament} and of the {Council} of 5 {June} 2019 on risk-preparedness in the electricity sector and repealing {Directive} 2005/89/{EC} (Jun. 2019).
\bibitem{koenker_regression_1978}
R.~Koenker, G.~Bassett, Regression {Quantiles}, Econometrica 46~(1) (1978) 33--50.
\newblock \href {https://doi.org/10.2307/1913643} {\path{doi:10.2307/1913643}}.
\bibitem{jiotsop-foze_electrical_2024}
W.~P. Jiotsop-Foze, A.~Hernández-del Valle, F.~Venegas-Martínez, Electrical load forecasting to plan the increase in renewable energy sources and electricity demand: a {CNN}-{QR}-{RTCF} and deep learning approach, Vol.~14, 2024, pp. 186--194.
\newblock \href {https://doi.org/10.32479/ijeep.15773} {\path{doi:10.32479/ijeep.15773}}.
\bibitem{guo_probabilistic_2024}
H.~Guo, B.~Huang, J.~Wang, Probabilistic load forecasting for integrated energy systems using attentive quantile regression temporal convolutional network, Advances in Applied Energy 14 (2024) 100165.
\newblock \href {https://doi.org/10.1016/j.adapen.2024.100165} {\path{doi:10.1016/j.adapen.2024.100165}}.
\bibitem{li_probabilistic_2024}
G.~Li, R.~Zhang, S.~Bu, J.~Zhang, J.~Gao, Probabilistic prediction-based multi-objective optimization approach for multi-energy virtual power plant, International Journal of Electrical Power \& Energy Systems 161 (2024) 110200.
\newblock \href {https://doi.org/10.1016/j.ijepes.2024.110200} {\path{doi:10.1016/j.ijepes.2024.110200}}.
\bibitem{waldmann_quantile_2018}
E.~Waldmann, Quantile regression: {A} short story on how and why, Statistical Modelling 18~(3-4) (2018) 203--218.
\newblock \href {https://doi.org/10.1177/1471082X18759142} {\path{doi:10.1177/1471082X18759142}}.
\bibitem{rosenblatt_remarks_1956}
M.~Rosenblatt, Remarks on {Some} {Nonparametric} {Estimates} of a {Density} {Function}, The Annals of Mathematical Statistics 27~(3) (1956) 832--837.
\newblock \href {https://doi.org/10.1214/aoms/1177728190} {\path{doi:10.1214/aoms/1177728190}}.
\bibitem{engle_autoregressive_1982}
R.~F. Engle, \href{https://www.jstor.org/stable/1912773?origin=crossref}{Autoregressive {Conditional} {Heteroscedasticity} with {Estimates} of the {Variance} of {United} {Kingdom} {Inflation}}, Econometrica 50~(4) (1982) 987.
\newblock \href {https://doi.org/10.2307/1912773} {\path{doi:10.2307/1912773}}.
\newline\urlprefix\url{https://www.jstor.org/stable/1912773?origin=crossref}
\bibitem{bollerslev_generalized_1986}
T.~Bollerslev, \href{https://linkinghub.elsevier.com/retrieve/pii/0304407686900631}{Generalized autoregressive conditional heteroskedasticity}, Journal of Econometrics 31~(3) (1986) 307--327.
\newblock \href {https://doi.org/10.1016/0304-4076(86)90063-1} {\path{doi:10.1016/0304-4076(86)90063-1}}.
\newline\urlprefix\url{https://linkinghub.elsevier.com/retrieve/pii/0304407686900631}
\bibitem{rigby_generalized_2005}
R.~A. Rigby, D.~M. Stasinopoulos, \href{https://academic.oup.com/jrsssc/article/54/3/507/7113027}{Generalized {Additive} {Models} for {Location}, {Scale} and {Shape}}, Journal of the Royal Statistical Society Series C: Applied Statistics 54~(3) (2005) 507--554.
\newblock \href {https://doi.org/10.1111/j.1467-9876.2005.00510.x} {\path{doi:10.1111/j.1467-9876.2005.00510.x}}.
\newline\urlprefix\url{https://academic.oup.com/jrsssc/article/54/3/507/7113027}
\bibitem{mayr_gamlss_2010}
A.~Mayr, N.~Fenske, B.~Hofner, T.~Kneib, M.~Schmid, \href{https://epub.ub.uni-muenchen.de/id/eprint/11938}{{GAMLSS} for high-dimensional data - a flexible approach based on boosting} (2010).
\newblock \href {https://doi.org/10.5282/UBM/EPUB.11938} {\path{doi:10.5282/UBM/EPUB.11938}}.
\newline\urlprefix\url{https://epub.ub.uni-muenchen.de/id/eprint/11938}
\bibitem{bousquet_extreme_2021}
N.~Bousquet, Extreme {Value} {Theory} with {Applications} to {Natural} {Hazards}: {From} {Statistical} {Theory} to {Industrial} {Practice}, Springer International Publishing AG, Cham, 2021.
\bibitem{renard_use_2007}
B.~Renard, M.~Lang, \href{https://linkinghub.elsevier.com/retrieve/pii/S0309170806001461}{Use of a {Gaussian} copula for multivariate extreme value analysis: {Some} case studies in hydrology}, Advances in Water Resources 30~(4) (2007) 897--912.
\newblock \href {https://doi.org/10.1016/j.advwatres.2006.08.001} {\path{doi:10.1016/j.advwatres.2006.08.001}}.
\newline\urlprefix\url{https://linkinghub.elsevier.com/retrieve/pii/S0309170806001461}
\bibitem{katz_statistics_2002}
R.~W. Katz, M.~B. Parlange, P.~Naveau, \href{https://linkinghub.elsevier.com/retrieve/pii/S0309170802000568}{Statistics of extremes in hydrology}, Advances in Water Resources 25~(8-12) (2002) 1287--1304.
\newblock \href {https://doi.org/10.1016/S0309-1708(02)00056-8} {\path{doi:10.1016/S0309-1708(02)00056-8}}.
\newline\urlprefix\url{https://linkinghub.elsevier.com/retrieve/pii/S0309170802000568}
\bibitem{martins_generalized_2000}
E.~S. Martins, J.~R. Stedinger, \href{https://agupubs.onlinelibrary.wiley.com/doi/10.1029/1999WR900330}{Generalized maximum‐likelihood generalized extreme‐value quantile estimators for hydrologic data}, Water Resources Research 36~(3) (2000) 737--744.
\newblock \href {https://doi.org/10.1029/1999WR900330} {\path{doi:10.1029/1999WR900330}}.
\newline\urlprefix\url{https://agupubs.onlinelibrary.wiley.com/doi/10.1029/1999WR900330}
\bibitem{ming_role_2024}
Z.~Ming, \href{https://cdn.misoenergy.org/20240926%20RA%20Risk%20Metric%20Workshop%20Item%2004%20Ming%20E3%20MISO%20Role%20of%20Metrics%20in%20Reliability650106.pdf}{The {Role} of {Metrics} in {Determining} a {Reliability} {Standard}} (Sep. 2024).
\newline\urlprefix\url{https://cdn.misoenergy.org/20240926%20RA%20Risk%20Metric%20Workshop%20Item%2004%20Ming%20E3%20MISO%20Role%20of%20Metrics%20in%20Reliability650106.pdf}
\bibitem{hochreiter_long_1997}
S.~Hochreiter, J.~Schmidhuber, \href{https://direct.mit.edu/neco/article/9/8/1735-1780/6109}{Long {Short}-{Term} {Memory}}, Neural Computation 9~(8) (1997) 1735--1780.
\newblock \href {https://doi.org/10.1162/neco.1997.9.8.1735} {\path{doi:10.1162/neco.1997.9.8.1735}}.
\newline\urlprefix\url{https://direct.mit.edu/neco/article/9/8/1735-1780/6109}
\bibitem{bai_empirical_2018}
S.~Bai, J.~Z. Kolter, V.~Koltun, \href{http://arxiv.org/abs/1803.01271}{An {Empirical} {Evaluation} of {Generic} {Convolutional} and {Recurrent} {Networks} for {Sequence} {Modeling}}, arXiv:1803.01271 (Apr. 2018).
\newblock \href {https://doi.org/10.48550/arXiv.1803.01271} {\path{doi:10.48550/arXiv.1803.01271}}.
\newline\urlprefix\url{http://arxiv.org/abs/1803.01271}
\bibitem{castillo_extreme_1988}
E.~Castillo, Extreme value theory in engineering, Statistical modeling and decision science, Academic Press, Boston, 1988.
\bibitem{majumdar_extreme_2019}
S.~N. Majumdar, A.~Pal, G.~Schehr, Extreme value statistics of correlated random variables: a pedagogical review (2019).
\newblock \href {https://doi.org/10.48550/ARXIV.1910.10667} {\path{doi:10.48550/ARXIV.1910.10667}}.
\bibitem{leadbetter_extremes_1983}
M.~R. Leadbetter, Extremes and {Related} {Properties} of {Random} {Sequences} and {Processes}, 1st Edition, Springer {Series} in {Statistics} {Ser}, Springer New York, New York, NY, 1983.
\bibitem{ganger_statistical_2014}
D.~Ganger, J.~Zhang, V.~Vittal, Statistical {Characterization} of {Wind} {Power} {Ramps} {Via} {Extreme} {Value} {Analysis}, IEEE Transactions on Power Systems 29~(6) (2014) 3118--3119.
\newblock \href {https://doi.org/10.1109/TPWRS.2014.2315491} {\path{doi:10.1109/TPWRS.2014.2315491}}.
\bibitem{wang_high_2018}
Z.~Wang, Z.~Liu, Y.~Sun, W.~Gao, C.~Gu, High {Impact} {Low} {Frequency} {Peak} {Load} {Analysis} using {Extreme} {Value} {Theory}, in: 2018 2nd {IEEE} {Conference} on {Energy} {Internet} and {Energy} {System} {Integration} ({EI2}), IEEE, Beijing, 2018, pp. 1--6.
\newblock \href {https://doi.org/10.1109/EI2.2018.8582202} {\path{doi:10.1109/EI2.2018.8582202}}.
\bibitem{breiman_bagging_1996}
L.~Breiman, Bagging predictors, Machine Learning 24~(2) (1996) 123--140.
\newblock \href {https://doi.org/10.1007/BF00058655} {\path{doi:10.1007/BF00058655}}.
\bibitem{kullback_information_1951}
S.~Kullback, R.~A. Leibler, On {Information} and {Sufficiency}, The Annals of Mathematical Statistics 22~(1) (1951) 79--86.
\newblock \href {https://doi.org/10.1214/aoms/1177729694} {\path{doi:10.1214/aoms/1177729694}}.
\bibitem{amari_natural_1998}
S.-i. Amari, Natural {Gradient} {Works} {Efficiently} in {Learning}, Neural Computation 10~(2) (1998) 251--276.
\newblock \href {https://doi.org/10.1162/089976698300017746} {\path{doi:10.1162/089976698300017746}}.
\bibitem{duan_ngboost_2020}
T.~Duan, A.~Avati, D.~Y. Ding, K.~K. Thai, S.~Basu, A.~Y. Ng, A.~Schuler, {NGBoost}: {Natural} {Gradient} {Boosting} for {Probabilistic} {Prediction} (Jun. 2020).
\newblock \href {https://doi.org/10.48550/arXiv.1910.03225} {\path{doi:10.48550/arXiv.1910.03225}}.
\bibitem{dey_extreme_2016}
D.~K. Dey, J.~Yan (Eds.), Extreme {Value} {Modeling} and {Risk} {Analysis}: {Methods} and {Applications}, Chapman and Hall/CRC, New York, 2016.
\newblock \href {https://doi.org/10.1201/b19721} {\path{doi:10.1201/b19721}}.
\bibitem{prescott_maximum_1980}
P.~Prescott, A.~T. Walden, Maximum likelihood estimation of the parameters of the generalized extreme-value distribution, Biometrika 67~(3) (1980) 723--724.
\newblock \href {https://doi.org/10.1093/biomet/67.3.723} {\path{doi:10.1093/biomet/67.3.723}}.
\bibitem{hosking_estimation_1985}
J.~R.~M. Hosking, J.~R. Wallis, E.~F. Wood, Estimation of the {Generalized} {Extreme}-{Value} {Distribution} by the {Method} of {Probability}-{Weighted} {Moments}, Technometrics 27~(3) (1985) 251--261.
\newblock \href {https://doi.org/10.2307/1269706} {\path{doi:10.2307/1269706}}.
\bibitem{coles_introduction_2001}
S.~Coles, \href{http://link.springer.com/10.1007/978-1-4471-3675-0}{An {Introduction} to {Statistical} {Modeling} of {Extreme} {Values}}, Springer {Series} in {Statistics}, Springer London, London, 2001.
\newblock \href {https://doi.org/10.1007/978-1-4471-3675-0} {\path{doi:10.1007/978-1-4471-3675-0}}.
\newline\urlprefix\url{http://link.springer.com/10.1007/978-1-4471-3675-0}
\bibitem{breiman_classification_2017}
L.~Breiman, J.~Friedman, R.~A. Olshen, C.~J. Stone, Classification and {Regression} {Trees}, Chapman and Hall/CRC, New York, 2017.
\newblock \href {https://doi.org/10.1201/9781315139470} {\path{doi:10.1201/9781315139470}}.
\bibitem{wald_tests_1943}
A.~Wald, Tests of {Statistical} {Hypotheses} {Concerning} {Several} {Parameters} {When} the {Number} of {Observations} is {Large}, Transactions of the American Mathematical Society 54~(3) (1943) 426--482.
\newblock \href {https://doi.org/10.2307/1990256} {\path{doi:10.2307/1990256}}.
\bibitem{dabernig_simultaneous_2017}
M.~Dabernig, G.~J. Mayr, J.~W. Messner, A.~Zeileis, \href{https://journals.ametsoc.org/doi/10.1175/MWR-D-16-0413.1}{Simultaneous {Ensemble} {Postprocessing} for {Multiple} {Lead} {Times} with {Standardized} {Anomalies}}, Monthly Weather Review 145~(7) (2017) 2523--2531.
\newblock \href {https://doi.org/10.1175/MWR-D-16-0413.1} {\path{doi:10.1175/MWR-D-16-0413.1}}.
\newline\urlprefix\url{https://journals.ametsoc.org/doi/10.1175/MWR-D-16-0413.1}
\bibitem{giacomet_distributional_2023}
C.~L. Giacomet, A.~C.~V. Ramos, H.~S.~D. Moura, T.~Z. Berra, Y.~M. Alves, F.~M. Delpino, J.~E. Farley, N.~R. Reynolds, J.~B. Alonso, T.~K.~A. Teibo, R.~A. Arcêncio, \href{https://doi.org/10.1186/s13690-023-01147-7}{A distributional regression approach to modeling the impact of structural and intermediary social determinants on communities burdened by tuberculosis in {Eastern} {Amazonia} – {Brazil}}, Archives of Public Health 81~(1) (2023) 135.
\newblock \href {https://doi.org/10.1186/s13690-023-01147-7} {\path{doi:10.1186/s13690-023-01147-7}}.
\newline\urlprefix\url{https://doi.org/10.1186/s13690-023-01147-7}
\bibitem{friederichs_forecast_2012}
P.~Friederichs, T.~L. Thorarinsdottir, \href{https://onlinelibrary.wiley.com/doi/abs/10.1002/env.2176}{Forecast verification for extreme value distributions with an application to probabilistic peak wind prediction}, Environmetrics 23~(7) (2012) 579--594.
\newblock \href {https://doi.org/10.1002/env.2176} {\path{doi:10.1002/env.2176}}.
\newline\urlprefix\url{https://onlinelibrary.wiley.com/doi/abs/10.1002/env.2176}
\bibitem{johnson_quantile-forest_2024}
R.~A. Johnson, \href{https://joss.theoj.org/papers/10.21105/joss.05976}{quantile-forest: {A} {Python} {Package} for {Quantile} {Regression} {Forests}}, Journal of Open Source Software 9~(93) (2024) 5976.
\newblock \href {https://doi.org/10.21105/joss.05976} {\path{doi:10.21105/joss.05976}}.
\newline\urlprefix\url{https://joss.theoj.org/papers/10.21105/joss.05976}
\bibitem{mcneil_quantitative_2015}
A.~J. McNeil, R.~Frey, P.~Embrechts, Quantitative risk management: concepts, techniques and tools, revised edition Edition, Princeton series in finance, Princeton University Press, Princeton Oxford, 2015.
\bibitem{velthoen_gradient_2022}
J.~Velthoen, C.~Dombry, J.-J. Cai, S.~Engelke, \href{http://arxiv.org/abs/2103.00808}{Gradient boosting for extreme quantile regression}, arXiv:2103.00808 (Dec. 2022).
\newblock \href {https://doi.org/10.48550/arXiv.2103.00808} {\path{doi:10.48550/arXiv.2103.00808}}.
\newline\urlprefix\url{http://arxiv.org/abs/2103.00808}
\bibitem{gnecco_extremal_2024}
N.~Gnecco, E.~M. Terefe, S.~Engelke, \href{http://arxiv.org/abs/2201.12865}{Extremal {Random} {Forests}}, arXiv:2201.12865 (Jan. 2024).
\newblock \href {https://doi.org/10.48550/arXiv.2201.12865} {\path{doi:10.48550/arXiv.2201.12865}}.
\newline\urlprefix\url{http://arxiv.org/abs/2201.12865}
\bibitem{ciabattoni_pjm_2024}
J.~Ciabattoni, B.~Chmielewski, {PJM} {Manual} 11: {Energy} and {Ancilary} {Services} {Market} {Operations} (Dec. 2024).
\bibitem{noauthor_data_nodate}
\href{https://dataminer2.pjm.com/list}{Data {Miner} 2}.
\newline\urlprefix\url{https://dataminer2.pjm.com/list}
\bibitem{hatch_pjm_2024}
K.~Hatch, {PJM} {Manual} 13: {Emergency} {Operations} (Dec. 2024).
\bibitem{farzan_toward_2015}
F.~Farzan, M.~A. Jafari, R.~Masiello, Y.~Lu, Toward {Optimal} {Day}-{Ahead} {Scheduling} and {Operation} {Control} of {Microgrids} {Under} {Uncertainty}, IEEE Transactions on Smart Grid 6~(2) (2015) 499--507, conference Name: IEEE Transactions on Smart Grid.
\newblock \href {https://doi.org/10.1109/TSG.2014.2368077} {\path{doi:10.1109/TSG.2014.2368077}}.
\bibitem{bahramara_day-ahead_2022}
S.~Bahramara, P.~Sheikhahmadi, A.~Mazza, G.~Chicco, Day-ahead self-scheduling from risk-averse microgrid operators to provide reserves and flexible ramping ancillary services, International Journal of Electrical Power \& Energy Systems 142 (2022) 108381.
\newblock \href {https://doi.org/10.1016/j.ijepes.2022.108381} {\path{doi:10.1016/j.ijepes.2022.108381}}.
\bibitem{guo_risk-averse_2022}
Y.~Guo, B.~Ming, Q.~Huang, Y.~Wang, X.~Zheng, W.~Zhang, Risk-averse day-ahead generation scheduling of hydro–wind–photovoltaic complementary systems considering the steady requirement of power delivery, Applied Energy 309 (2022) 118467.
\newblock \href {https://doi.org/10.1016/j.apenergy.2021.118467} {\path{doi:10.1016/j.apenergy.2021.118467}}.
\end{thebibliography}