Towards using utility data to quantify how investments would have increased the wind resilience of distribution systems

Extracted source text · arXiv 2306.06526

\documentclass{IEEEtran}
\IEEEoverridecommandlockouts
\usepackage{amsmath, amssymb, amsfonts}
\usepackage{algorithmic, graphicx, textcomp, xcolor}
\usepackage{tabularx}
\usepackage{array}
\newcolumntype{S}{>{\hsize=.5\hsize\linewidth=\hsize}X}
\newcolumntype{L}{>{\hsize=1.5\hsize\linewidth=\hsize}X}    %the sum of all columns widths must add up to a multiple of X


\usepackage[
style=ieee,doi=false,isbn=false,url=false,eprint=false,maxnames=4
]{biblatex}
\addbibresource{refs.bib} %Import the bibliography file

\begin{document}

\title{Towards using utility data to quantify how investments would have increased the wind resilience of distribution systems
%Measuring increased resilience from wind resilience investment using distribution utility outage data\\
}

\author{Arslan Ahmad%,~\IEEEmembership{Student Member~IEEE} 
\qquad\qquad Ian Dobson%,~\IEEEmembership{Fellow~IEEE}
\\Iowa State University, June 2023
\thanks{A. Ahmad and I. Dobson are with the Electrical and Computer Engineering Department,
Iowa State University,
Ames, IA, USA. 
dobson@iastate.edu. Funding from the U.S. Government, the U.S. Department of State, IIE, and USEFP via the Fulbright program and USA NSF grants 2153163 and 2228757 is gratefully acknowledged.}
}

\maketitle

\begin{abstract}
We quantify resilience with metrics extracted from the historical outage data that is routinely recorded by many distribution utilities.
The outage data is  coordinated  with wind data to relate average outage rates in an area to wind speed measured at a nearby weather station. 
A past investment in wind hardening would have reduced the outage rates, and the effect of this on metrics can be calculated by sampling a reduced number of the historical outages and recomputing the metrics. 
This quantifies the impact that the hardening would have had on customers. 
This is a tangible way to relate an investment in wind resilience to the benefits it would have had on the lived experience of customers that could help make the case for the investment to the public and regulators.
We also quantify the impact of earlier or faster restoration on customer metrics and compare this to the impact of investment in hardening. Overall this is a new and straightforward approach to quantify resilience and justify resilience investments to stakeholders that is directly driven by utility data. The approach driven by data avoids complicated models or modeling assumptions.
\end{abstract}

\begin{IEEEkeywords}
Power distribution systems, outages, resilience, metrics, fragility, data, weather, wind
\end{IEEEkeywords}

%ISSUES

\section{Introduction}

Resilience generally addresses the response of power systems to extreme weather events, as well as other unusual high stresses such as earthquakes, fires, and epidemics. 
There are many overlapping frameworks and definitions of resilience \cite{StankovicPS22,PanteliPROCIEEE17,NanRESS17} and the most concise definition is ``Power system resilience is the ability to limit the extent, severity, and duration of system degradation following an extreme event." \cite{CIGREELECTRA19}. There remains much scope for practically quantifying resilience \cite{JiPROCIEEE17} and for justifying investments that improve resilience. 
In particular, this paper proposes a data-driven approach to quantify the resilience of a distribution system to wind and to help justify investments in hardening the system or speeding up its restoration. Wind-related hazards due to storms and hurricanes are one of the most significant hazards for overhead distribution system; the effects of high winds include tree falls and flying debris as well as direct impacts such as pole toppling and conductor galloping.

Investment decisions upgrading distribution systems or their restoration consider many factors, such as cost, reliability, load, distributed generation, deployment of crews and materials, and many engineering and community constraints. 
In order to also consider resilience in these decisions, it is desirable to quantify the benefits of investments that increase resilience and communicate those benefits to utilities, communities, and regulators. 
One useful but complex approach is to make detailed models of the extreme weather, its impact on the distribution system,  the restoration process, and the impact on customers and then use these models to estimate the future benefits of a proposed upgrade. 
This model-based approach is challenging because of the extensive approximations and assumptions needed to make practical models of the entire process. Nevertheless, there has been considerable progress in using these models as reviewed in section \ref{litreview}. 
The purpose of this paper is to open up another, complementary approach driven directly by utility data that can also help to quantify and communicate the benefits of investments that increase resilience to strong winds. 

\begin{figure}[h]
    \centering
    \includegraphics[width=0.48\textwidth]{images/flowDiagram.png}
    \caption{Quantifying the benefits of wind-hardening resilience investments.}
    \label{fig:flowDiagram}
\end{figure}


We briefly outline the new data-driven approach in Fig.~\ref{fig:flowDiagram} and as follows:
Consider an area inside the distribution system that is close to a weather station measuring wind speed. 
We process the area outage data and the wind speed together to obtain an ``area outage rate curve" that describes how the mean outage rate of the area increases with wind speed. 
The area outage rate curve quantifies the resilience of the area with respect to the measured wind speed. 
A previous investment in hardening the area would have shifted the area outage rate curve and reduced the area outage rates.
We can go back to the historical outage data and sample a reduced number of outages from it to match the reduced outage rates. 
This samples the historical outages that the investment would have caused. Computing resilience metrics for the historical outages and comparing these to the improved resilience metrics for the sampled outages quantifies the effect that the investment would have had on the area and its customers.
Moreover, we also quantify the impact of improved restoration:
Investment in restoration can make the repairs start earlier or increase the rate at which repairs are completed.
We change the historical restoration times to correspond to these improvements and recalculate the metrics to quantify their impacts.
This brief outline is elaborated throughout the paper, but we start by discussing the value of the new approach in justifying resilience investments.

In addition to the routine reluctance to pay more for improved electrical infrastructure, resilience investments are particularly hard to justify because they concern rarer large events that will recur, but at an indefinite time in the future\footnote{Note that reliability investments already address the common failures.}. 
There are indeed technical difficulties in estimating the impact of rare events, but arguably at least as important for practically improving resilience are the difficulties in communicating the impact of rare events and the benefits of investing to reduce their impact. 
The model-based approach can estimate with some approximations and assumptions the projected future benefits of a resilience investment. The data-driven approach can estimate the benefits that the investment would have had if the investment had previously been made. 
In some ways, the data-driven approach can be more persuasively tangible because it is related to people's lived past experience. 
For example, it could be persuasive to say that the community would have saved 20\% fewer customer minutes of outages over the past 10 years, or even more specifically that a particular, extreme wind event (such as the upper midwest USA derecho in August 2020 that caused $\sim$11 billion dollars of damage) would have had 20\% fewer customer minutes out.


The overall paper contribution is 
showing the feasibility of a new, entirely data-driven method of quantifying  
distribution system resilience to wind, particularly the change in standard metrics that would have occurred if overall investments in hardening or earlier or faster restoration had been made. 
There are no modeling assumptions,
the data is readily available to utilities with an outage management system,
and the computations are fast and relatively straightforward.
The new quantification of resilience benefits of investments can be tangible to stakeholders and help support the case for investments.




After reviewing the literature in section~\ref{litreview}, section~\ref{DataSection} 
describes the outage and weather data used. Section \ref{outageratecurves} describes 
the area outage rate curves 
that quantify the wind resilience from data, and how they can be shifted to represent hardening.
Section \ref{MetricsSection} explains how to extract resilience events from data and calculate their metrics.
Section \ref{SamplingSection} describes the sampling that represents the hardening and 
Section \ref{restoration} shows how improved restoration is represented.
Results quantifying the improvement in metrics due to the hardening and improved restoration 
are presented in Section \ref{results}.
Sections \ref{outageratecurveDerivationSection} and \ref{superevents} give technical details of constructing area outage curves and tracking events after sampling.
Finally, the paper contributions and conclusions are given in section \ref{conclusions}.
The paper elaborates and builds on the initial work presented in the MS thesis \cite{AhmadMS23}.


\section{Literature review}
\label{litreview}

Since high winds are a significant hazard for overground power distribution systems \cite{AhmadMS23,climate22}, there is substantial work studying the resilience of these systems to the wind.

%\subsection{model based approaches to quantify resilience}

We begin by briefly surveying a variety of methods that build and use models to quantify distribution system resilience.
These methods form a useful complementary approach to our new data-driven quantification of resilience.
Xu \cite{XuQuanta08} defines a calculator for hurricanes
that calculates the cost and benefits of distribution system undergrounding or hardening given 
utility estimates of input data. The modeling includes exponential fragility models for poles, power law fragility models for span damage, 
simulated hurricanes, and restoration times estimated from crew availability.
Ma \cite{MaPS18} formulates system hardening, the impacts of extreme weather, and minimizing load shedding as a multi-level mixed-integer linear program to find an optimal hardening 
strategy. Arif \cite{ArifPS18} co-optimizes distribution system operation and repair crew routing for outage restoration after extreme weather events using a two-stage stochastic mixed integer linear program.
%Ma \cite{MaPESGM19} extends the optimization to include modeling of all phases from preparation and hardening to restoration.
Tan \cite{TanPS18} finds an optimal hardening and repair sequence to minimize the expected energy not served using mixed integer 
linear programming and associated convex relaxations and heuristics.
Tan \cite{TanPS19} finds optimal repair sequencing when there is large-scale damage by solving a scheduling problem 
using approximations to linear programming and accounting for
multiple faults obstructing the power to the same customer.
Ouyang \cite{OuyangSS14} models the resilience of Harris County, Texas, to hurricane Ike using exponential fragility curves, a DC power flow network model, and high-level models of restoration resources and sequencing.
Ciapessoni \cite{CiapessoniENERGIES21} evaluates the power grid's resilience in a mountainous area to the combined effects of wind, snow, and trees with a risk-based model.
Poudel \cite{PoudelSJ19} simulates for a distribution of wind conditions the improvement in resilience risk of planned improvements using measures such as value at risk of energy not served.
Wei \cite{WeiAM16} analyzes resilience to particular severe hurricanes with time-varying Poisson processes that model distribution system outage and repair processes as a queue. 


%as in Zapata \cite{ZapataTandDCE08} to model reliability and 
%Brown \cite{BrownQuanta09} estimates costs and benefits of hurricane hardening.


%\subsection{data-based resilience for distribution systems}

We now survey methods that analyze dependencies in distribution system data to describe 
wind resilience. Much of the analysis is for specific hurricanes.
Davidson \cite{DavidsonNHR03} studies resilience to hurricanes in the Carolinas using utility outage data, a combination of interpolated and simulated hurricane wind data, and land cover and rain data. 
Liu \cite{LiuRESS08} constructs a spatial generalized linear mixed model of the number of hurricane and ice storm outages in zip codes as a function of weather, protection device density, and spatial data. 
Reed \cite{ReedSJ09} studies the resilience to wind-induced damage in Hurricane Katrina. 
Jaech \cite{JaechPS19} estimates individual component repair times using a gamma distribution with parameters predicted using neural networks trained on utility outage records.
%For example, Jaech \cite{JaechPS19} predicts individual component outage restoration times and customer hours lost with a neural network, and
%Chow \cite{ChowPD96} analyzes the contributions of timing, faults, protection, outage types and weather to individual distribution system component restoration times.
%Some papers use gamma distributions to fit outage times or restoration times \cite{JaechPS19,CarringtonPS21}
Carrington \cite{CarringtonPS21} establishes methods for extracting events from utility outage data, computes standard resilience metrics for the events, and estimates restoration times.
%Related methods for transmission systems are in \cite{EkishevaPMAPS22}.
Cerrai \cite{cerraiACCESS19} and Kezunovic \cite{kezunovicHICSS18} use machine learning methods based on multiple decision trees and logistic regression respectively to combine weather prediction, vegetation, outage and other data to predict the probability of storm outages in small areas of the distribution system. 


Now we review relevant work extracting fragility curves from wind and outage data for use in resilience models. Dunn \cite{DunnNHR18} develops empirical fragility curves for overhead lines from 11 kV to 132 kV in the UK during wind storms, defined as winds exceeding 38 mph. 
The fragility curves are fit with power laws, and relate the number of faults per line length in areas of $\sim$2000 km$^2$ to the reanalyzed maximum wind gust over the storm duration in the areas.
Reed \cite{ReedWEIA08} analyzes the fragility and outage duration of an urban distribution system for four different wind storms. Fragility curves show the fraction of affected feeder length as a function of the peak storm wind gust squared. 
Bjarnado \cite{BjarnadoIS13} describes lognormal fragility curves for pole design subject to hurricane risk as well as reviewing deterministic pole design with safety factors. 
Murray \cite{MurrayPMAPS14} correlates reanalyzed wind gust data with transmission system faults in the UK to give an exponential fragility curve for transmission lines. 

Reliability addresses performance averaged over the year rather than resilience, but we acknowledge the extensive and useful tradition of evaluating distribution system reliability with steady-state Markov models \cite{BillintonAllanBook} in which extreme weather is modeled with additional Markov states \cite{BillintonGTD06}. 

\section{Outage data and weather data} \label{DataSection}
Two different datasets are used, one with outage data recorded by a distribution utility in the USA and the other containing NOAA (National Oceanic and Atmospheric Administration) wind data for weather stations in that utility's service area.
Both datasets cover the same time span of six years.
The outage dataset contains 32\,278 individual outages and has a one-minute resolution; i.e., all outages that occurred within one minute are labeled with the same timestamp.
Each outage entry in the dataset corresponds to an outage of a component in the power distribution system and includes the location coordinates of that component, the number of customers affected during the outage, the starting and ending time of the outages, and cause codes.\footnote{149 outage records with missing location information are removed. Further details about data cleaning and pre-processing are given in \cite{AhmadMS23}.}
%The cause codes are assigned to each outage by the utility to indicate the reason behind it, such as weather, trees, equipment failure, and accident.

Each record in the weather dataset provides the average hourly wind speed, and various other weather details. 
%of different observed weather parameters, including wind speed, wind gust speed, wind direction, precipitation, sky conditions, relative humidity, and monthly summaries; however, 
Only the average hourly wind speed data is used here.
There is data from multiple weather stations in the utility's service area.
With the outage and weather stations' geographical locations known, each outage is associated with the weather station closest to the outage. 
This divides the distribution system into multiple areas, where each area contains all the outages closest to the weather station associated with that area. This paper analyzes the outages in two of these areas, as shown in Fig.~\ref{fig:outageLocations}.
Area 1 and area 2 contain 7876 and 12\,715 outages, respectively.

The typical measured wind speeds at the weather station in area 2 are systematically slower than the typical measured wind speeds at the weather station for area 1. Differences in these two measurements are expected: 
Weather station 2 measures wind at 5 feet from the ground in a field surrounded by woods, whereas weather station 1 measures wind at 33 feet from the ground in a large flat area with no trees. The weather stations are 13 miles apart.

\begin{figure}[ht]
    \centering
    \includegraphics[width=0.48\textwidth]{images/outagesMap.png}
    \caption{Geographical location of outages and the associated weather stations in two areas of a distribution system.}
    \label{fig:outageLocations}
\end{figure}


\section{Area outage rate curves}
\label{outageratecurves}

This section explains area outage rate curves, how they quantify wind resilience, and how shifting them represents the 
effect of distribution system hardening.
%Section \ref{outageratecurveDerivationSection} gives the details of calculating area outage rate curves from the outage and wind data.

\subsection{Quantifying wind resilience with area outage rate curves}

Each area of the distribution system is associated with its weather station.
The area outage rate curve specifies the mean outage rate of the area $\overline{F}(v)$ as a function of the wind speed $v$ measured at the weather station.
The mean outage rate generally increases with wind speed, as shown by Figs. \ref{fig:MBExponentialModel} and \ref{fig:PKExponentialModel}. 
The dots indicating the mean outage rates at integer wind speeds in Fig. \ref{fig:MBExponentialModel} and \ref{fig:PKExponentialModel} are calculated from the wind and outage data for areas 1 and 2; the details are given in section \ref{outageratecurveDerivationSection}. The curves in Fig. \ref{fig:MBExponentialModel} and \ref{fig:PKExponentialModel} are exponential fits of the form 
\begin{align}
\label{eq:exponentialEquation}
   \overline{F}(v)=ae^{bv}
\end{align}
The exponential fit uses the Levenberg-Marquardt method (also known as the damped least-squares method) with a 99\% confidence level for parameters and predictions. The parameter values obtained for the fit are $a=1.44\times10^{-7}$ and $b=0.6$ for area 1, and $a=0.006$ and $b=0.48$ for area 2.
The wind station in area 2 measures slower wind speeds than the wind station in area 1. This is expected as discussed in section \ref{DataSection}.
\begin{figure}[ht]
    \centering
    \includegraphics[width=0.48\textwidth]{images/PKExponentialModel.pdf}
    \caption{Area outage rate curve of area 1. Dots are the mean outage rate at each wind speed from data, and the curve is an exponential fit.}
    \label{fig:MBExponentialModel}
\end{figure}
\begin{figure}[ht]
    \centering
    \includegraphics[width=0.48\textwidth]{images/MBExponentialModel.pdf}
    \caption{Area outage rate curve of area 2. Dots are the mean outage rate at each wind speed from data, and the curve is an exponential fit.}
    \label{fig:PKExponentialModel}
\end{figure}

The outage rate curve describes the resilience of the area with respect to the wind measurement at the weather station.
The outage rate is low, except that it increases sharply for higher wind speeds. Other authors report a dependence of outage rate data on wind speed of a similar form and use exponential \cite{BrownQuanta09,XuQuanta08,MurrayPMAPS14} or power law \cite{DunnNHR18,XuQuanta08} fits to describe this dependence.

\subsection{Hardening shifts the area outage rate curves}
\label{hardening}

Overhead power line components such as poles are designed to withstand their rated wind speed. Hardening upgrades or reinforces the components. 
The overall effect of the hardening is that the same mean outage rate can be achieved at a higher wind speed; that is, the hardening shifts the outage rate curve to the right, as shown in 
Fig.~\ref{fig:shiftedCurvesOverallMB}.\footnote{A right-shift is also used to model the failure rate data for hardened transmission structures in \cite[Figs. 5-4 and 5-5]{BrownQuanta09}} For example, replacing poles rated for 60 mph wind with poles rated at $(60+x)$ mph would shift the outage rate curve $\overline{F}(v)$ right by $x$ mph so that the new outage rate curve is $ \overline{F}_{new}(v)=\overline{F}(v-x)$. 
Since the mean outage rate generally increases with wind speed, $\overline{F}_{new}(v)<\overline{F}(v)$ so that the hardening reduces the mean outage rate at wind speed $v$. 
This reduction in the outage rate is implemented with sampling in section~\ref{SamplingSection}. In the case of the exponential area outage rate curve (\ref{eq:exponentialEquation}), the reduction in outage rate takes the simple form of multiplying the outage rate by the same factor $e^{-bx}$ at all wind speeds:
\begin{align}
   \overline{F}_{new}(v)=\overline{F}(v-x)=ae^{b(v-x)} = \overline{F}(v)e^{-bx}  \label{eq:exponentialModelShift}
\end{align}

\begin{figure}[ht]
    \centering
    \includegraphics[width=0.48\textwidth]{images/shiftedCurvesOverallMB.pdf}
    \caption{Comparison of original and shifted area outage rate curve of area 2 after 1 mph hardening to wind hazard.}
    \label{fig:shiftedCurvesOverallMB}
\end{figure}

\subsection{Comparing area outage rate curves and fragility curves}

Fragility curves for wind describe the probability of component failure or failure per kilometer of line as a function of wind speed \cite{DunnNHR18, MurrayPMAPS14,ReedWEIA08}.
Area outage rate curves have some similarities and differences with fragility curves, so it is interesting to compare them. 
Area outage rate curves give the mean outage rate of an area as a function of the measurement of wind speed at a particular weather station, whereas fragility curves give the probability of failure of a component as a function of the wind speed (at least conceptually) at that component. The wind speed at any given component in the area is correlated with, but different than the wind speed at any particular weather station.
%\footnote{For example, the correlation between wind measured at two nearby points is used when siting wind turbines to relate measurements for a limited time at the proposed site with many years of measurement at a weather station cite MPE methods.}.
The area contains many types of components that can cause an outage (poles, lines, insulators, etc.) and these components can also differ in manufacturer, age, elevation, topography, local environment, tree cover, and condition and are subject to different wind conditions. 
Since the area outage rate curve is directly obtained from historical data, it incorporates all these types and variations, and describes the aggregated response of the area in terms of outages with respect to the wind speed at the particular weather station. 
One notable difference is in the use of the two curves: area outage rate curves bypass any component modeling to directly describe the aggregated resilience of the entire distribution system area with respect to a particular wind measurement, whereas fragility curves are used in models of components to design that component or to compute the resilience of many similar components. Indeed, it is not appropriate to directly substitute an area outage rate curve for a component fragility curve in models.

%The mean outage rate is related to the failure probability under some assumptions: if one models the distribution system area as $N$ similar components under wind stress that is constant over a time period and assumes independent component failures, then the mean outage rate of the area is proportional to $N$ times the individual component failure probability.
%One aspect is that an area outage rate curve involves time differently than a fragility curve: A fragility curve gives a probability of failure if a given wind speed is applied to the component without considering the time at or over which the wind speed is applied, whereas an outage rate inherently applies over a time interval. To address this, fragility curves can be formulated as depending on the maximum wind speed over an event cite Dunn.
%To enable use in models and predictions, fragility curves can be derived from data to give failure probabilities per km of line (\cite{DunnNHR18, DunnICASP2015})cite reed.

%The area outage rate curves are overall similar to the fragility curves introduced by Dunn et al. (\cite{DunnNHR18, DunnICASP2015}), but there are many detailed differences in the processing and some differences in interpretation/application.
%The fragility curves by Dunn et al. show a relationship between the average number of faults/1000km in a distribution system during wind storm events (defined by wind speeds $\geq$ 17/ms).

\section{Events, processes and resilience metrics}  \label{MetricsSection}
A necessary step in the data processing groups outages into resilience events and then calculates several metrics for each event.
The particular metrics we use are among the typical resilience metrics proposed and explained in references such as \cite{PanteliPS17FLEP,RaoufiSUSTAIN20}, allowing for the observation that in real data the theoretically successive phases of resilience usually overlap \cite{CarringtonPS21}.
%To do this we apply the methods of \cite{CarringtonPESGM20}. This section summarizes these methods, relying on \cite{CarringtonPESGM20} for a detailed description. 
The focus on events, particularly the larger ones, and the metrics for events instead of average performance over a year, make this a resilience analysis. 

To define the resilience events and automatically extract them from the distribution system data we use the method in \cite{CarringtonPESGM20,CarringtonPS21}. %\footnote{Related methods for transmission systems are in \cite{EkishevaPMAPS22}.} 
The start of an event is defined by an initial outage that occurs when all components are operational, and the end of the same event is defined by the first subsequent time when all the components are restored. 
We write $n$ for the number of outages in an event.
If we write $o_1$ for the start time of the first outage and $r_n$ for the time of the last restore, then the event occurs over the time interval $[o_1,r_n]$.
Two example events are shown in Fig.~\ref{fig:events}.

\begin{figure}[h]
    \centering
    \includegraphics[width=0.48\textwidth]{images/eventfigure.pdf}
    \caption{An event with one outage and an event with 3 outages. Above the time axis shows each outage start time (open circle) and restore time (dot). Below the time axis is the performance curve $P(t)$ for each event.}
    \label{fig:events}
\end{figure}

Performance curves that track in time the negative of the cumulative number of unrestored outages or customers or other quantities are routinely used in studies of resilience to track the progress in time of resilience events \cite{PanteliPS17FLEP,NanRESS17,PanteliPROCIEEE17,StankovicPS22,PoudelSJ19}.
Accordingly, we define the component performance curve $P(t)$ as the negative of the cumulative number of unrestored outages in an event.
($P(t)$ is also the cumulative number of restores at time $t$ minus the cumulative number of outages at time $t$ \cite{CarringtonPS21}.)
Component performance curves for two events are shown in Fig.~\ref{fig:events}.
$P(t)$ decrements by one when there is an outage and increments by one when there is a restore.
In particular, $P(t)$ is initially zero, the event starts at $o_1$ when the cumulative number of failures $P(t)$ first decrements from zero, and ends at $r_n$ when $P(t)$ increases to return to zero. 
The events group together the successive outages that have some overlap in duration. 
Events in our distribution utility data (2944 total events in area 1 and 3706 in area 2) are of all sizes, ranging from a single outage that is restored without involving any other outages to the largest event with more than 100 overlapping outages. 

In an event with $n$ outages, we write $o_1\le o_2\le ...\le o_n$ for the outage times in the order in which they occur and $r_1\le r_2\le ...\le r_n$ for the restore times in the order in which they occur.
The outages happen in the time interval $[o_1,o_n]$ and the restores happen in the time interval $[r_1,r_n]$, and typically in real data the restores start before the outages end so that these time intervals overlap.
We write $c_k$ for the number of customers outaged at the $k$th outage.

The component performance curve $P(t)$ tracking the number of unrestored outages easily generalizes to a customer performance curve $P^{\rm cust}(t)$ that tracks the number of unrestored customers: $P^{\rm cust}(t)$ is the negative of the cumulative number of unrestored customers in an event.

It is now straightforward to give formulas for the resilience metrics that we evaluate for each event:
\begin{itemize}
  \item {\sl event size} = number of outages $=n$
  \item {\sl outage hours} = area under performance curve\\ $=r_1-o_1+r_2-o_2+...+r_n-o_n=-\int_{o_1}^{r_n}P(t)dt$
  \item {\sl event duration} $=r_n-o_1$
  \item {\sl time to first restore} $=r_1-o_1$
  \item {\sl restore duration} $=r_n-r_1$
  \item {\sl restore rate} $=n/(r_n-r_1)$
  \item {\sl outage rate} $=n/(o_n-o_1)$
  \item {\sl customers out} $= c_1+c_2+...+c_n$
  \item {\sl customer hours} = area under customer performance curve $=c_1(r_1-o_1)+c_2(r_2-o_2)+...+c_n(r_n-o_n)$ $=-\int_{o_1}^{r_n}P^{\rm cust}(t)dt$
\end{itemize}
The two expressions for {\sl outage hours}, or for {\sl customer hours}, are shown to be equal in \cite{DobsonArxiv23}.
Note that dividing the {\sl customer hours} for an event by the number of customers gives the contribution of that event to SAIDI, assuming for the larger events that major event days are included in SAIDI. 

\section{Outage sampling to get the average metrics for reduced outage rates}  \label{SamplingSection}
This section describes the sampling from the historical outages to select a reduced number of outages that represents hardening.
The resilience metrics are recalculated for many such samples and then averaged to find the average improvements in the metrics.
This metric calculation is applied separately to small, medium, and large events.

Suppose that the mean outage rate $\overline{F}(v)$ at wind speed $v$ is calculated from the $k$ outages $\left \{ e_{1},...,e_{k} \right \}$. 
According to section \ref{hardening}, a shift in the area outage rate curve gives the new outage rate $\overline{F}_{\rm new}(v)$ at wind speed $v$. 
To realize this reduced outage rate, we randomly sample $k_{new}$ outages from $\left \{ e_{1},...,e_{k} \right \}$ where, in general, and writing $\lceil\cdot\rceil$ for rounding up to the next integer,
\begin{align}
   k_{new} = \bigg\lceil k\frac{\overline{F}_{\rm new}(v)}{\overline{F}(v)}\bigg\rceil
   \label{eq:OutagesSubsetSizeNew}
\end{align}
But in our case of exponential outage rate curves, (\ref{eq:OutagesSubsetSizeNew}) simplifies using (\ref{eq:exponentialModelShift}) to 
\begin{align}
   k_{new} = \lceil k e^{-bx}\rceil \label{eq:OutagesSubsetSizeNewexp}
\end{align}

The sampling method is simple sampling with replacement and it is applied at each wind speed to obtain a new set of outages at all wind speeds that realizes the new area outage rate curve and the effect of the hardening.
We calculate the new resilience metrics for the new set of outages.
It is convenient to write $M$ for one of these resilience metrics, and $M_{\rm new}^{(1)}$ for the metric evaluated on the new set of outages at all wind speeds.
The entire sampling and metric evaluation procedure is then repeated $m$ times to obtain the new metrics $M_{\rm new}^{(1)}$, $M_{\rm new}^{(2)}$, ..., $M_{\rm new}^{(m)}$.
Finally, the average new metric is computed as 
\begin{align}
   \overline{M}_{\rm new} = \frac{1}{m}\sum_{i=1}^m M_{\rm new}^{(i)}\label{eq:AverageM}
\end{align}

An explanation for this procedure is that while the shift in the outage rate curve determines the new reduced number of outages $k_{\rm new}$ at each wind speed, it does not determine {\sl which} outages are to be omitted when realizing this reduced number of outages.
That is, we do not know which outages at each wind speed will be removed by the hardening.
Therefore we compute the average new metric $\overline{M}_{\rm new}$ for random samples of the reduced number of outages. 

One complication is that the sampling can sometimes remove outages from an event in such a way that the event splits into smaller events.
This complication is handled with super events as explained in section \ref{superevents}.

For our calculations, the number of repetitions of the sampling procedure is chosen as $m=2000$ to ensure that the confidence interval $\overline{M}_{\rm new}\pm 0.01$ contains the true value of the mean metric with probability 99\% or greater.
$m=2000$ is obtained as follows: Since the distributions of the sampled metrics $M_{\rm new}$ are observed to be approximately normal, the half width $d$ of a $99\%$ confidence interval for the mean $\overline{M}_{\rm new}$ satisfies 
\begin{align}   
    d\leq t_{0.005,m-1}\frac{s}{\sqrt{m}}           \label{eq:standardError}
\end{align}
where $s$ is the sample standard deviation of the metric samples %$M_{\rm new}^{(1)}$, $M_{\rm new}^{(2)}$, ..., $M_{\rm new}^{(m)}$ 
and $t_{0.005,m-1}$ is the 99.5\% percentile of the Student-$t$ distribution with $m-1$ degrees of freedom. 
We take $d=0.01$ and increase $m$ until (\ref{eq:standardError}) is satisfied for each metric.

There is a clear pattern in the data of far more smaller events and much fewer large events. 
For example, Fig.~\ref{fig:eventSizePlotMB} shows the empirical probability distribution of {\sl event size} for area~1 on a log-log scale. 
This pattern affects the processing of the results, because if one averages all the results together, the smaller events will dominate the average.
To address this, and particularly because resilience must have some focus on the large events, we divide the events into small events (1 or 2 outages), medium events (3 to 15 outages), and large events (16 or more outages). 
Area 1 has 2386 small events, 526 medium events, and 32 large events, and area 2 has 2845 small events, 773 medium events and 88 large events.
Average metrics for small, medium, or large events can distinguish the resilience performance for these different sizes of events, while still having enough large events to give usable estimates of the average metrics for large events.

\begin{figure}[htb]
    \centering
    \includegraphics[width=0.48\textwidth]{images/eventSizePlotMB.pdf}
    \caption{Empirical distribution of {\sl event size} in area 1 (log-log scale)}
    \label{fig:eventSizePlotMB}
\end{figure}

\section{Representing earlier or faster restoration}
\label{restoration}

%We have already discussed the impacts of 10\% wind hardening investments on the resilience metrics of the distribution system in previous sections. In this section, we discuss the impacts of investments to improve service restoration and compare them with the hardening investments.

This section represents the effects of improved restoration by modifying the restore times of the historical data.
Recall that in an event with $n$ outages, we write $r_1\le r_2\le ...\le r_n$ for the restore times in the order in which they occur. The outage times of the components that are restored in this restore order are written as $o_{\pi(1)}, o_{\pi(2)},...,o_{\pi(n)}$. The components do not usually outage in the order in which they are restored, so the $\pi$ function permutes the order to account for this.

We represent the improved restoration in two ways.
First, the repair can start earlier by providing more resources for identifying, locating, and automatically resolving faults; this includes investing in more sensors, switches, communications, 
meters, and reclosers as well as more crews to inspect the lines and clear debris.
Let the change in start time be specified by $t_{\rm earlier}$, then the new restore times for the event are 
\begin{align}
    r_k^{\rm new}=\max\{r_k-t_{\rm earlier},o_{\pi(k)}\}, \quad k=1,...,n.
    \label{rkearlier}
\end{align}
Taking the maximum with $o_{\pi(k)}$ in (\ref{rkearlier}) limits the new restore time so that $r_k^{\rm new}\ge o_{\pi(k)}$; restoration of a component must occur after its outage.
%Limiting the shortening time to the component repair time $\rho_k$ ensures that the component repair times remain nonnegative.

Second, the rate of restoration can be increased and the restoration duration decreased by investing in more repair crews, better stocks of spare parts, and better route scheduling.
Let the faster restore duration be specified by multiplying by a factor $c_{\rm faster}<1$.
Then the restore duration of the $k$th restore $r_k-r_1$ is reduced by a factor of $c_{\rm faster}$, as long as the new restore time occurs after its corresponding outage:
\begin{align}
    r_k^{\rm new}=\max\{r_1+(r_k-r_1)c_{\rm faster},o_{\pi(k)}\}, \  k=1,...,n.
    \label{eq:rnewrate}
\end{align}
%Note that in the common case of the new restore times not being limited by the component restore times, this investment decreases the restore duration $r_n-r_1$ by a factor of $d$ and increases the average restore rate $n/(r_n-r_1)$ by a factor $1/d$.


\section{Results}
\label{results}

\begin{table*}[!t]
    \renewcommand{\arraystretch}{1.1}
    \caption{Average metrics for small, medium, large events and their changes with hardening or earlier or faster restoration}
    \label{tab:resultsArea1}
    \centering
    \begin{tabular}{l c c c c c c c c c c c c}
    \multicolumn{13}{c}{AREA 1}\\
        \hline
&\multicolumn{3}{c}{Base Case Events}&\multicolumn{3}{c}{Change with Hardening}&\multicolumn{3}{c}{Change with Earlier Restore}&\multicolumn{3}{c}{Change with Faster Restore}\\[-2pt]
Resilience Metric&small&medium&large&small&medium&large&small&medium&large&small&medium&large\\
        \hline
        {\sl event size}              & 1.22    & 4.64    & 78.91   & -10.0\%    & -10.0\%   & -10.0\%   & 0\%     & 0\%        & 0\%      & 0\%      & 0\%      & 0\%      \\
        {\sl outage hours}            & 3.06    & 17.33   & 2084    & -10.0\%    & -10.0\%   & -10.0\%   & -74.8\% & -58.6\%    & -10.0\%  & -0.6\%   & -5.7\%   & -10.0\%  \\
        {\sl event duration}          & 2.72    & 8.22    & 46.62   & -8.9\%     & -4.5\%    & -1.2\%    & -68.2\% & -27.5\%    & -5.5\%   & -0.7\%   & -3.9\%   & -4.4\%  \\
        {\sl time to first restore}   & 2.40    & 2.97    & 2.95    & -7.6\%     & 1.7\%     & 2.3\%     & -75.9\% & -65.1\%    & -69.0\%  & 0\%      & 0\%      & 0\%      \\
        {\sl restore duration}        & 0.32    & 5.25    & 43.67   & -19.0\%    & -10.7\%   & -2.3\%    & -10.0\% & -6.3\%     & -1.2\%   & -6.2\%   & -6.1\%   & -4.7\%   \\
        {\sl customers out}           & 45.26   & 257.93  & 5610    & -9.9\%     & -10.0\%   & -10.1\%   & 0\%     & 0\%        & 0\%      & 0\%      & 0\%      & 0\%      \\
        {\sl customer hours}          & 85.88   & 680.83  & 95304   & -9.9\%     & -9.9\%    & -10.1\%   & -80.5\% & -68.2\%    & -14.1\%  & -0.5\%   & -5.4\%   & -9.1\%   \\[2pt]
        \multicolumn{13}{c}{AREA 2}\\
        \hline
        {\sl event size}              & 1.25    & 4.75    & 62.48   & -10.0\%    & -10.0\%   & -10.0\%   & 0\%     & 0\%        & 0\%      & 0\%      & 0\%      & 0\%      \\
        {\sl outage hours}            & 2.71    & 15.28   & 1142    & -10.0\%    & -10.0\%   & -10.0\%   & -67.8\% & -51.1\%    & -10.0\%  & -0.5\%   & -4.6\%   & -10.0\%  \\
        {\sl event duration}          & 2.39    & 7.51    & 37.67   & -8.8\%     & -4.5\%    & -1.3\%    & -61.7\% & -22.7\%    & -4.8\%   & -0.6\%   & -3.1\%   & -4.0\%   \\
        {\sl time to first restore}   & 2.10    & 2.66    & 2.56    & -7.4\%     & 1.6\%     & 3.0\%     & -68.6\% & -56.8\%    & -62.3\%  & 0\%      & 0\%      & 0\%      \\
        {\sl restore duration}        & 0.28    & 4.85    & 35.11   & -19.0\%    & -10.8\%   & -2.9\%    & -10.5\% & -3.9\%     & -0.6\%   & -4.8\%   & -4.8\%   & -4.3\%   \\
        {\sl customers out}           & 50.74   & 242.98  & 4084    & -10.0\%    & -10.1\%   & -10.0\%   & 0\%     & 0\%        & 0\%      & 0\%      & 0\%      & 0\%      \\
        {\sl customer hours}          & 85.51   & 547.28  & 58700   & -10.0\%    & -10.0\%   & -10.1\%   & -74.0\% & -61.9\%    & -12.0\%  & -0.3\%   & -4.7\%   & -9.0\%   \\
        \hline
        &\multicolumn{3}{c}{all time quantities in hours}&\multicolumn{9}{c}{small events have 1--2 outages, medium events have 3--15 outages, large events have $\ge$16 outages}\\
    \end{tabular}
\end{table*}

This section presents a case study of the impacts of a hardening and improved restorations on the resilience metrics for areas 1 and 2 of the distribution system. 

\subsection{Base case resilience metrics}
The base case is the historical outages without any modifications.
Table \ref{tab:resultsArea1} shows the base case average values of the resilience metrics for small events (1 or 2 outages), medium events (3--15 outages), and large events ($\ge$16 outages).
Considering the different sizes of events separately and with some special attention to the large events is needed for this quantification of resilience, as explained at the end of section~\ref{SamplingSection}. 
As expected, all the metrics (except {\sl time to first restore}) clearly show the increased impact on customers as the event size increases from small to medium to large. 
The average resilience metrics show that area 1 has greater customer impacts than area 2 for large events.

\subsection{Change in metrics due to hardening}
The hardening for each area with respect to wind is represented by an increased mile per hour wind rating, which gives a percentage reduction in the outage rate (see section \ref{hardening}) that is implemented by sampling a reduced number of outages (see section \ref{SamplingSection}). 
For the case study, it is convenient to consider a hardening that gives a 10\% reduction in outage rate for both areas.
This 10\% reduction in the outage rate corresponds to 0.18 mph wind hardening for area 1 and 0.22 mph wind hardening for area 2.
The 10\% reduction in outage rate is implemented by sampling 10\% fewer outages, so that the hardening reduces the average {\sl event size} (number of outages) by exactly 10\%, as confirmed in Table \ref{tab:resultsArea1}.
The hardening also reduces the average {\sl outage hours} by exactly 10\%. This result follows from a resilience metric formula in \cite{DobsonArxiv23}.\footnote{
For each event, \cite[(17)]{DobsonArxiv23} gives {\sl outage hours}
$=n \overline{\rho}=n \left(\frac{1}{n}\sum_{k=1}^n \rho_k\right)$,
 which is the number of outages $n$ times the average component restore time $\overline{\rho}$ in the event. 
 Averaging over all the samples with $n$ reduced by 10\% leaves the expectation of the average of $\overline{\rho}$ over all the samples equal to $\overline{\rho}$. Therefore the average {\sl outage hours} reduces by exactly 10\%.}

Table \ref{tab:resultsArea1} shows that the hardening decreases the {\sl customers out} and {\sl customer hours} by approximately 10\% for events of all sizes in both areas. 
However, the reductions in the average duration metrics are less than 10\%, except for the {\sl restore duration} of small events.
When we sample the reduced number of outages, one or more outages get removed from events randomly.
Depending on exactly which outage is removed, the {\sl event duration} and {\sl restore duration} metrics either decrease or remain the same.
The {\sl restore duration} of small-size events reduces by more than 10\% because the {\sl restore duration} drops to zero when an outage is randomly removed for an event having only two outages.
Removing an outage has a larger percentage impact on durations in a small event than in a large event, so that the average decrease is smaller for large events. 
In particular, it is notable that the hardening which gives close to a 10\% average reduction in {\sl customer hours} reduces the average {\sl event duration} for large events by less than 2\%.
The {\sl time to first restore} metric can decrease, remain the same, or increase when outages are removed from an event. The effects of outage removal on the duration metrics are further discussed in \cite{AhmadMS23}.
% Moreover, the decrease in these three metrics gets smaller from small to large-size events. Because as the event size increases, the probability of a critical outage getting selected randomly decreases. A critical outage, in this case, is the outage that changes these three metrics if removed from the event. A more detailed explanation of this phenomenon is given in (reference to the thesis).

\subsection{Change in metrics due to earlier or faster restoration}
Table \ref{tab:resultsArea1} also presents the change in metrics for the two types of improved restoration.
The earlier restoration time is specified by the shortening time $t_{\rm earlier}$ in (\ref{rkearlier}), and 
the faster restoration is specified by the improvement factor $c_{\rm faster}$ in (\ref{eq:rnewrate}). 
It is convenient in order to facilitate comparisons to select $t_{\rm earlier}$ and $c_{\rm faster}$ for each area so that the average {\sl outage hours} for large events decrease by exactly 10\%.
In particular, we select $t_{\rm earlier}=2.84$ hour and $c_{\rm faster}=0.9385$ for area 1 and $t_{\rm earlier}=1.92$ hour and $c_{\rm faster}=0.9522$ for area 2.

The earlier and faster restorations do not remove any outages, so the {\sl event size} and {\sl customers out} metrics stay the same.

%in the case of earlier restorations and the improvement factor $c_{\rm faster}$ in the case of faster restorations are selected to have an exact 10\% decrease in the outage-hours of events in the large size category. 
%This choice is made to make the changes due to improved restorations comparable to those due to hardening. 
%Moreover, since we are more interested in studying the impacts of large-size events, the values are selected for large events. The same values are applied to the small and medium size events categories to see the impacts.

%In the case of earlier restorations, a shortening time of 115.2 minutes and 170.2 minutes gives a 10\% reduction in the mean outage-hours of large events of areas 1 and 2, respectively. 
%Therefore if investments were made such that the restoration process could be started earlier by these values, then we would have seen a 10\% decrease, on average, in the outage-hours of large-size events in both areas. 

The earlier restoration in (\ref{rkearlier}) reduces restoration times by $t_{\rm earlier}$, but with a limitation so that restoration of each component does not occur before it outages. 
This limitation for some restores causes the average {\sl time to first restore} to reduce, but by less than $t_{\rm earlier}$.
For large events, the average {\sl time to first restore} reduces from 2.95 hrs to 0.92 hrs and from 2.56 hrs to 0.96 hrs in area 1 and area 2, respectively. 
%It is because we can only start the restore process as soon as the outage process starts and cannot start it before that, and therefore for some of the events, we cannot shorten the time exactly by $t_{\rm earlier}$.
In the absence of the limitation in (\ref{rkearlier}), all restoration times are reduced by $t_{\rm earlier}$ and the {\sl restore duration} stays the same.
However, the limitation affects some of these restorations, and these can include the first and last restoration of an event, and this results in the average {\sl restore duration} decreasing.
%Similarly, the change in restore duration should be zero as, in the ideal case, we only shift the restore process to an earlier time, keeping the restore duration the same. However, we can only shift the restore process by an amount equal to the duration (repair time) of the last restored outage $r_n$, and also, we cannot shift it more than the duration of the first restored outage $r_1$. Therefore the restore duration either increases or decreases depending on $t_{\rm earlier}\lesseqgtr r_n$ and $t_{\rm earlier}\lesseqgtr r_1$. An overall decrease in the mean restore duration of both areas shows that the shift gets limited by $r_1$ more often on average.

%For the faster restoration case, an improvement factor of 0.9522 and 0.9385 gives a 10\% reduction in the mean outage-hours of large events of areas 1 and 2, respectively. In other words, if investments were made to increase the restore rates by approximately 5\% and 6.6\%, then those would have resulted in a 10\% reduction in the mean outage-hours of large-size events of area 1 and area 2, respectively. 

The faster restoration in (\ref{eq:rnewrate}) speeds up the restoration by the factor $c_{\rm faster}$ over the duration of the restoration, but with a limitation so that restoration of each
component does not occur before it outages. 
In the absence of the limitation in (\ref{eq:rnewrate}), the {\sl restore duration} would decrease by 6.2\% and 4.8\% corresponding to the $c_{\rm faster}$ values for area 1 and area 2, but we see in Table \ref{tab:resultsArea1} less decrease in some of the mean {\sl restore durations} due to the limitation.
The {\sl time to the first restore} is unaffected by the faster restoration. 
The change in the average {\sl customer hours} quite closely follows the change in the average {\sl outage hours}. 
The faster restoration affects the small events differently because they have only one or two outages.
The events with only one outage remain unchanged as their {\sl restore duration} is zero. Also, a large number of two-outage events have zero {\sl restore duration} when both outages restore at the same time.
Therefore, those small events also remain unchanged. Consequently, only a small proportion of small events see improvements due to faster restoration, which explains the very small changes in the average metrics other than {\sl restore duration}.

\subsection{Comparing hardening and improved restoration}
Two overall options are available for power system resilience investments. One is to invest in hardening and the other is to invest in improved restoration.
The hardening invests in the infrastructure, whereas improved restorations invest in the crews and their resources and automated actions.
Our results confirm the general observation that hardening reduces the {\sl event size} (number of outages) while affecting event durations less, whereas improved restorations decrease the outage durations but do not affect the {\sl event size}. 

The results in table \ref{tab:resultsArea1} show that a hardening decreasing {\sl outage hours} and {\sl event size} by 10\% also decreases {\sl customer hours} and {\sl customers out} by approximately 10\%, irrespective of small, medium or large events.
The {\sl customer hours} are particularly important in assessing the impact of power outages on consumers.
On the other hand, the improved restorations to achieve a 10\% reduction in the mean {\sl outage hours} for large events also provide almost the same percentage decrease in {\sl customer hours}. However, unlike hardening, the {\sl event size} and {\sl customers out} remain unchanged.
So the customers would still face power outages but get restored more quickly.

All these results show how different overall investments would have changed the various resilience metrics and customer impacts. This quantifies the various benefits that the investments would have made to the utility and their customers.

\section{Constructing area outage rate curves }  
\label{outageratecurveDerivationSection}
This section explains how to align and process the outage and wind data to construct the area outage rate curves. 

Since the outage times are recorded to the nearest minute and the wind speeds are recorded hourly\footnote{Eighteen outages with a time difference of more than 201 minutes and low wind speeds are omitted from further analysis. Some of the wind speed data is at 15 minute or more than one hour intervals.},
we need to interpolate the wind speeds.
Let $V(t)$ be the piecewise linear interpolation of the wind data as shown in Fig.~\ref{fig:windSpeedInterpolation}. 
$V^{-1}(v)=\left \{ t\mid V(t)=v \right \}$ defines the set of time instants with wind speed $v$.
Fig.~\ref{fig:windSpeedInterpolation} shows an example of $V^{-1}(v)=\left \{ t_{1},t_{2} \right \}$ when the wind speed value is $v$ at times $t_{1}$ and $t_{2}$. 

The outage times are quantized to the nearest minute. 
Therefore, if $t$ is an exact number of minutes, then the outage rate $R(t)$ per minute is given by the number of outages recorded in the area at minute $t$. 
Further, if $t$ is not an exact number of minutes, the outage rate at $t$ is given by $R(\mathrm{round}[t])$, where the rounding is to the nearest minute.

The outage rate curve $\overline{F}(v)$ is the mean outage rate at wind speed $v$; that is, the mean of the outage rates over all the times at which the wind speed was $v$:
\begin{align} \label{eq:fragilityEquation}
    \overline{F}(v)=\frac{1}{\left | V^{-1}(v) \right |}\sum_{t\in V^{-1}(v)}^{}R(\mathrm{round}[t])
\end{align}

\begin{figure}[ht]
    \centering
    \includegraphics[width=0.48\textwidth]{images/windSpeedInterpolation.pdf}
    \caption{Linear interpolation of wind speed data }
    \label{fig:windSpeedInterpolation}
\end{figure}

%It is characteristic of the higher wind data that it gets sparser as the wind speed increases.
%For moderate and higher speeds, the wind data distribution is well approximated by Weibull distributions [give parameters for are 1 and area 2 ??] and the high wind speed data lies in the upper tail of these distributions.

%[HOW DO WE OR SHOULD WE SHOW THE INDIVIDUAL OUTAGE RATES AS IN Fig. \ref{fig:windBubblePlotPK} or a different figure]
% The result is shown in Fig. \ref{fig:windBubblePlotPK}. The individual outage rates corresponding to each wind speed are also shown in this figure to visualize the overall distribution of outage rates for each wind speed. This plot shows that the data gets sparser as the wind speed increases. There are two factors behind this trend, the low probability of occurrence of higher wind speeds and the low probability of higher outage rates.

%\begin{figure}[ht]
%    \centering
%    \includegraphics[width=0.48\textwidth]{images/windBubblePlotPK.png}
 %   \caption{Individual Outage Rates at each wind speed level in Area-1}
 %   \label{fig:windBubblePlotPK}
%\end{figure}

%Consistent with \cite{OuyangSS12} more routine hazards impact resilience more significantly than hurricane hazards due to their higher frequency and randomness.????

\section{Sampling and super events}
\label{superevents}

Since the sampling removes outages, the remaining outages that were in the same event before sampling may not all be overlapping after sampling and so can sometimes split into two or more events. 
We call the set of events arising after sampling in this way from one event before sampling a ``super event", and this section explains super events.
For example, consider the timeline plot of a typical event $E=\{e_1$, $e_2$, $e_3$, $e_4$, $e_5$, $e_6$, $e_7$, $e_8$, $e_9\}$ with 9 outages before sampling as shown in Fig.~\ref{fig:resilienceEventTimeline}.  
If the sampling removes $e_1$, then the remaining outages still form one event and the super event is $\{\{e_2$, $e_3$, $e_4$, $e_5$, $e_6$, $e_7$, $e_8$, $e_9\}\}$. 
However, if the sampling removes $e_4$ and $e_5$ then the remaining outages form two events since the system is fully restored when outage $e_3$ is restored and the super event is $\{\{e_1, e_2, e_3\}, \{e_6, e_7, e_8, e_9\}\}$. 

The changes in the sizes and numbers of events due to sampling cause problems when the metrics of events before and after sampling are compared: 
Basic to the analysis is the classification of events by size based on their number of outages, and the variable reduction in event size, and especially events splitting into multiple smaller events, interferes with tracking the effect of the sampling on event metrics and disrupts the effect of the sampling on the categories of small, medium, and large events, since the events can change categories after sampling. 
These problems are resolved by keeping track of all the events arising after sampling from one event before sampling in a super event, and appropriately defining the metric of a super event as follows:

Consider a metric $M$ that can be evaluated on an event $E$ as $M[E]$, and a super event $\{E_1,E_2,...,E_p\}$ that has events $E_1,E_2,...,E_p$ after sampling. Then for the metrics 
{\sl event size} (number of outages),
{\sl outage hours},
{\sl restore duration},
{\sl number of customers out}, and
{\sl customer hours}, we define the metric evaluated on the super event as 
\begin{align}
    M[\{E_1,E_2,...,E_p\}]=
    M[E_1]+M[E_2]+...+M[E_p]\notag
\end{align}
and for the the metrics 
{\sl restore rate},
{\sl outage rate}, and
{\sl time to first restore}, we replace summation by the average to define
\begin{align}
    M[\{E_1,E_2,...,E_p\}]=\mbox{$\frac{1}{p}$}
    (M[E_1]+M[E_2]+...+M[E_p])\notag
\end{align}
Events with only one outage can disappear if that outage is removed by sampling. In this case, the super event is the empty set $\{\,\}$ and all the metrics evaluate to zero.

\begin{figure}[htb]
    \centering
    \includegraphics[width=0.48\textwidth]{images/resilienceEventTimeline.pdf}
    \caption{Timeline plot of a medium size resilience event with 9 overlapping outages shown by line segments. For each line segment, the open circle is the outage start time and the dot is the outage restore time.}
    \label{fig:resilienceEventTimeline}
\end{figure}






\section{Conclusions}
\label{conclusions}

This paper combines historical outage and weather data to construct area outage rate curves to quantify the resilience of areas of a distribution system to wind. 
An investment hardening the distribution system would have shifted the area outage rate curves and reduced the outage rates, and the effect of this on resilience metrics is quantified by sampling a reduced number of the historical outages and recalculating the resilience metrics. 
The effect of an improvement in restoration times is also quantified by advancing or speeding up the historical restoration and recalculating the metrics.
The resilience metrics include the {\sl event size} (number of outages), durations, rates, and {\sl customer hours} evaluated on resilience events of different sizes. 
These data-driven calculations quantify the impact on customers that previous investments would have had.

Overall, we initiate a new approach towards resilience quantification.
Specific contributions and attributes of this new approach are:
\begin{itemize}
    \item Quantifying the impact on customers that a resilience investment would have made in the past gives a novel and credible way to justify the benefits of the investment that can be tangible to utilities, communities, and regulators, because it clearly shows how the lived past experience of customers would have been improved.
    This is significant since effective ways to justify resilience investments to stakeholders are essential for practically implementing resilience. Quantifying the effect that the investment would have made in the past complements and augments justifications for resilience investments that rely on projections into the future with models. 
    \item Modifying historical data is an entirely new way to quantify resilience and resilience investments that calculates the changes in standard resilience metrics from the effects that the investments would have had. This approach directly driven by data has clear advantages in realism in accounting for all the conditions that the power system experienced over the period of observation, including variations in space and time in weather, load, upgrades, operating procedures and restoration policies, and component design, location, conditions and maintenance. No modeling assumptions are made. 
    \item We construct area outage rate curves that quantify the wind resilience of an area of a distribution system directly from data by describing how the mean outage rate of the area increases as a specific nearby wind measurement increases. The area outage rate curves have a similar form as component fragility curves, but describe the empirical aggregate area response in terms of outages rather than the response to wind experienced at specific components of the distribution system. 
%     Area outage rate curves are not suitable to be directly inserted into a component fragility model.
    \item The overall effect of resilience investments are simply represented by an earlier or faster restoration or by a hardening that increases wind strength by a given number of miles per hour. This enables a novel and credible comparison of investment in hardening versus investment in better restoration in terms of customer impact. 
    \item %Wind hazard hardening by a given number of miles per hour shift area outage rate curves and reduce outage rates. The effect that the reduced outage rates would have had are implemented by sampling subsets of outages from the historical data. Resilience metrics of small, medium, and large events are calculated from the sampled outages and compared with the original resilience metrics to assess the impact of the hardening investments. 
    The technical aspects of the calculations include: \\(a) the segregation of resilience events into small, medium, and large events to get a meaningful assessment of the resilience of events of different sizes with metrics, (b) super events to track the metrics of events that split into smaller events when outages are removed, and \\ (c) leveraging recent work \cite{CarringtonPS21} that automatically extracts events of all sizes from utility outage data and calculates a range of metrics for each event. 
    %The calculated impact of the hardening or faster restoration can be analytically confirmed for some metrics.
    \item The method is limited to a net reduction in historical outages; it does not synthesize additional new outages. Note that this limitation does not rule out the incorporation of all future effects. For example, the effect of a future increase in the average wind speed can still be represented, as long as it is offset by sufficient hardening so that the net outage rate decreases. 
    Also the effect of a percentage increase in the rate of events is straightforward to evaluate because the metrics per event are unchanged, and any metrics that accumulate over a time period increase by the same percentage. The feasibility of these extensions is significant since wind speeds and storm frequencies are expected to increase with climate change.
    \item The outage data and weather data required to use this approach are easily available to any distribution utility with an outage management system, and the computations are relatively straightforward and fast.
\end{itemize}


There is promising scope for extending the methods of this paper in future work: Other wind data could be tested and evaluated.
Other hazards such as flood or icing could be considered and outage cause code information could be related to the hazards and leveraged.
Detailed model-based approaches could be used to directly link specific engineering improvements to the overall changes in hardening and faster restoration that model the investments in this paper. 
If cost estimates and probability estimates of rarer events can be improved, then the risk and improvements to risk could be quantified.

This paper has established the feasibility and useful characteristics of a new approach to quantifying resilience and the benefits of investments in resilience from historical data, and we are confident that further developments can follow.



\renewcommand*{\UrlFont}{\rmfamily}
\renewcommand*{\bibfont}{\footnotesize}
\printbibliography %Prints bibliography

\end{document}

% $ biblatex auxiliary file $
% $ biblatex bbl format version 3.2 $
% Do not modify the above lines!
%
% This is an auxiliary file used by the 'biblatex' package.
% This file may safely be deleted. It will be recreated by
% biber as required.
%
\begingroup
\makeatletter
\@ifundefined{ver@biblatex.sty}
  {\@latex@error
     {Missing 'biblatex' package}
     {The bibliography requires the 'biblatex' package.}
      \aftergroup\endinput}
  {}
\endgroup


\refsection{0}
  \datalist[entry]{none/global//global/global}
    \entry{StankovicPS22}{article}{}
      \name{author}{1}{}{%
        {{hash=ff83f780a562dd4a92931635a9da988a}{%
           family={Stankovic\bibnamedelima et\bibnamedelima al.},
           familyi={S\bibinitperiod\bibinitdelim e\bibinitperiod\bibinitdelim a\bibinitperiod},
           given={A.},
           giveni={A\bibinitperiod}}}%
      }
      \strng{namehash}{ff83f780a562dd4a92931635a9da988a}
      \strng{fullhash}{ff83f780a562dd4a92931635a9da988a}
      \strng{bibnamehash}{ff83f780a562dd4a92931635a9da988a}
      \strng{authorbibnamehash}{ff83f780a562dd4a92931635a9da988a}
      \strng{authornamehash}{ff83f780a562dd4a92931635a9da988a}
      \strng{authorfullhash}{ff83f780a562dd4a92931635a9da988a}
      \field{sortinit}{1}
      \field{sortinithash}{4f6aaa89bab872aa0999fec09ff8e98a}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{journaltitle}{IEEE Trans. Power Systems}
      \field{note}{early access}
      \field{title}{Methods for analysis and quantification of power system resilience}
      \field{year}{2022}
    \endentry
    \entry{PanteliPROCIEEE17}{article}{}
      \name{author}{1}{}{%
        {{hash=b2cf5adc002e8112ed4ce8d39d3fcb0d}{%
           family={Panteli\bibnamedelima et\bibnamedelima al.},
           familyi={P\bibinitperiod\bibinitdelim e\bibinitperiod\bibinitdelim a\bibinitperiod},
           given={M.},
           giveni={M\bibinitperiod}}}%
      }
      \strng{namehash}{b2cf5adc002e8112ed4ce8d39d3fcb0d}
      \strng{fullhash}{b2cf5adc002e8112ed4ce8d39d3fcb0d}
      \strng{bibnamehash}{b2cf5adc002e8112ed4ce8d39d3fcb0d}
      \strng{authorbibnamehash}{b2cf5adc002e8112ed4ce8d39d3fcb0d}
      \strng{authornamehash}{b2cf5adc002e8112ed4ce8d39d3fcb0d}
      \strng{authorfullhash}{b2cf5adc002e8112ed4ce8d39d3fcb0d}
      \field{sortinit}{1}
      \field{sortinithash}{4f6aaa89bab872aa0999fec09ff8e98a}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{journaltitle}{Proc. IEEE}
      \field{number}{7}
      \field{title}{Power systems resilience assessment: hardening and smart operational enhancement}
      \field{volume}{105}
      \field{year}{2017}
    \endentry
    \entry{NanRESS17}{article}{}
      \name{author}{2}{}{%
        {{hash=8f8263788f198bdac27589687edde5a6}{%
           family={Nan},
           familyi={N\bibinitperiod},
           given={C.},
           giveni={C\bibinitperiod}}}%
        {{hash=eb73ffd3f8fd74e03d3dcb606d197faf}{%
           family={Sansavini},
           familyi={S\bibinitperiod},
           given={G.},
           giveni={G\bibinitperiod}}}%
      }
      \strng{namehash}{97ed16cb820708d85aee3feaf36d0906}
      \strng{fullhash}{97ed16cb820708d85aee3feaf36d0906}
      \strng{bibnamehash}{97ed16cb820708d85aee3feaf36d0906}
      \strng{authorbibnamehash}{97ed16cb820708d85aee3feaf36d0906}
      \strng{authornamehash}{97ed16cb820708d85aee3feaf36d0906}
      \strng{authorfullhash}{97ed16cb820708d85aee3feaf36d0906}
      \field{sortinit}{1}
      \field{sortinithash}{4f6aaa89bab872aa0999fec09ff8e98a}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{journaltitle}{Reliability Engineering and System Safety}
      \field{title}{A quantitative method for assessing resilience of interdependent infrastructures}
      \field{volume}{157}
      \field{year}{2017}
      \field{pages}{35\bibrangedash 53}
      \range{pages}{19}
    \endentry
    \entry{CIGREELECTRA19}{article}{}
      \name{author}{1}{}{%
        {{hash=0af2059c6b681f4ddd4d19ed95e6d82a}{%
           family={{CIGRE Working Group C4.47}},
           familyi={C\bibinitperiod}}}%
      }
      \strng{namehash}{0af2059c6b681f4ddd4d19ed95e6d82a}
      \strng{fullhash}{0af2059c6b681f4ddd4d19ed95e6d82a}
      \strng{bibnamehash}{0af2059c6b681f4ddd4d19ed95e6d82a}
      \strng{authorbibnamehash}{0af2059c6b681f4ddd4d19ed95e6d82a}
      \strng{authornamehash}{0af2059c6b681f4ddd4d19ed95e6d82a}
      \strng{authorfullhash}{0af2059c6b681f4ddd4d19ed95e6d82a}
      \field{sortinit}{2}
      \field{sortinithash}{8b555b3791beccb63322c22f3320aa9a}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{journaltitle}{Electra}
      \field{title}{Defining power system resilience}
      \field{volume}{306}
      \field{year}{2019}
      \field{pages}{32\bibrangedash 34}
      \range{pages}{3}
    \endentry
    \entry{JiPROCIEEE17}{article}{}
      \name{author}{3}{}{%
        {{hash=0336023bb7f0ba2acdfee3ea6cb9a44a}{%
           family={Ji},
           familyi={J\bibinitperiod},
           given={Chuanyi},
           giveni={C\bibinitperiod}}}%
        {{hash=cdcc947b44bb9b97365a7b9fb939aad6}{%
           family={Wei},
           familyi={W\bibinitperiod},
           given={Yun},
           giveni={Y\bibinitperiod}}}%
        {{hash=de7b0e512fcdd087f5750837f7c7cb9f}{%
           family={Poor},
           familyi={P\bibinitperiod},
           given={H.\bibnamedelimi Vincent},
           giveni={H\bibinitperiod\bibinitdelim V\bibinitperiod}}}%
      }
      \strng{namehash}{904c3a3caf8420c444ae47b89586f327}
      \strng{fullhash}{904c3a3caf8420c444ae47b89586f327}
      \strng{bibnamehash}{904c3a3caf8420c444ae47b89586f327}
      \strng{authorbibnamehash}{904c3a3caf8420c444ae47b89586f327}
      \strng{authornamehash}{904c3a3caf8420c444ae47b89586f327}
      \strng{authorfullhash}{904c3a3caf8420c444ae47b89586f327}
      \field{sortinit}{3}
      \field{sortinithash}{ad6fe7482ffbd7b9f99c9e8b5dccd3d7}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{journaltitle}{Proc. IEEE}
      \field{number}{7}
      \field{title}{Resilience of Energy Infrastructure and Services: Modeling, Data Analytics, and Metrics}
      \field{volume}{105}
      \field{year}{2017}
      \field{pages}{1354\bibrangedash 1366}
      \range{pages}{13}
      \verb{doi}
      \verb 10.1109/JPROC.2017.2698262
      \endverb
    \endentry
    \entry{AhmadMS23}{thesis}{}
      \name{author}{1}{}{%
        {{hash=21d73b282329dfd1f20694a366b33e76}{%
           family={Ahmad},
           familyi={A\bibinitperiod},
           given={Arslan},
           giveni={A\bibinitperiod}}}%
      }
      \list{institution}{1}{%
        {Iowa State University}%
      }
      \strng{namehash}{21d73b282329dfd1f20694a366b33e76}
      \strng{fullhash}{21d73b282329dfd1f20694a366b33e76}
      \strng{bibnamehash}{21d73b282329dfd1f20694a366b33e76}
      \strng{authorbibnamehash}{21d73b282329dfd1f20694a366b33e76}
      \strng{authornamehash}{21d73b282329dfd1f20694a366b33e76}
      \strng{authorfullhash}{21d73b282329dfd1f20694a366b33e76}
      \field{sortinit}{4}
      \field{sortinithash}{9381316451d1b9788675a07e972a12a7}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{title}{Quantifying the Impact of Investments on Distribution System Wind Resilience Using Area Outage Rate Curves}
      \field{type}{MS thesis}
      \field{year}{2023}
    \endentry
    \entry{climate22}{report}{}
      \name{author}{1}{}{%
        {{hash=261d5b3f78c9e96d8a5083e7d89d8996}{%
           family={{Climate Central}},
           familyi={C\bibinitperiod}}}%
      }
      \list{institution}{1}{%
        {Climate Central}%
      }
      \list{publisher}{1}{%
        {Climate Central}%
      }
      \strng{namehash}{261d5b3f78c9e96d8a5083e7d89d8996}
      \strng{fullhash}{261d5b3f78c9e96d8a5083e7d89d8996}
      \strng{bibnamehash}{261d5b3f78c9e96d8a5083e7d89d8996}
      \strng{authorbibnamehash}{261d5b3f78c9e96d8a5083e7d89d8996}
      \strng{authornamehash}{261d5b3f78c9e96d8a5083e7d89d8996}
      \strng{authorfullhash}{261d5b3f78c9e96d8a5083e7d89d8996}
      \field{sortinit}{5}
      \field{sortinithash}{20e9b4b0b173788c5dace24730f47d8c}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{month}{09}
      \field{title}{Surging Power Outages and Climate Change}
      \field{type}{techreport}
      \field{year}{2022}
      \verb{urlraw}
      \verb https://assets.ctfassets.net/cxgxgstp8r5d/73igUswSfOhdo7DUDVLwK7/bb0a4e95e1d04457e56106355a1f74b9/2022PowerOutages.pdf
      \endverb
      \verb{url}
      \verb https://assets.ctfassets.net/cxgxgstp8r5d/73igUswSfOhdo7DUDVLwK7/bb0a4e95e1d04457e56106355a1f74b9/2022PowerOutages.pdf
      \endverb
    \endentry
    \entry{XuQuanta08}{report}{}
      \name{author}{2}{}{%
        {{hash=0e4b43cc503b4583d4023aa2af56f914}{%
           family={Xu},
           familyi={X\bibinitperiod},
           given={Le},
           giveni={L\bibinitperiod}}}%
        {{hash=2e7c4d1d38f7906188ea79dadf997b1d}{%
           family={Brown},
           familyi={B\bibinitperiod},
           given={Richard},
           giveni={R\bibinitperiod}}}%
      }
      \list{institution}{1}{%
        {prepared by Quanta Technology for Florida Electric Utilities}%
      }
      \strng{namehash}{d22c0206dbdd8fd27d7024efbe1f50b4}
      \strng{fullhash}{d22c0206dbdd8fd27d7024efbe1f50b4}
      \strng{bibnamehash}{d22c0206dbdd8fd27d7024efbe1f50b4}
      \strng{authorbibnamehash}{d22c0206dbdd8fd27d7024efbe1f50b4}
      \strng{authornamehash}{d22c0206dbdd8fd27d7024efbe1f50b4}
      \strng{authorfullhash}{d22c0206dbdd8fd27d7024efbe1f50b4}
      \field{sortinit}{6}
      \field{sortinithash}{b33bc299efb3c36abec520a4c896a66d}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{title}{Undergrounding Assessment Phase 3 Report: Ex Ante Cost and Benefit Modeling}
      \field{year}{2008}
    \endentry
    \entry{MaPS18}{article}{}
      \name{author}{3}{}{%
        {{hash=c7dbe653e34afac9875652532c74d5ef}{%
           family={Ma},
           familyi={M\bibinitperiod},
           given={Shanshan},
           giveni={S\bibinitperiod}}}%
        {{hash=74c79d6b9bd7252bf08c34a3342be609}{%
           family={Chen},
           familyi={C\bibinitperiod},
           given={Bokan},
           giveni={B\bibinitperiod}}}%
        {{hash=e9735f5160a24d53df53e9f5b75435d3}{%
           family={Wang},
           familyi={W\bibinitperiod},
           given={Zhaoyu},
           giveni={Z\bibinitperiod}}}%
      }
      \strng{namehash}{f170fe32a2ea53edf7b7190240e50c2d}
      \strng{fullhash}{f170fe32a2ea53edf7b7190240e50c2d}
      \strng{bibnamehash}{f170fe32a2ea53edf7b7190240e50c2d}
      \strng{authorbibnamehash}{f170fe32a2ea53edf7b7190240e50c2d}
      \strng{authornamehash}{f170fe32a2ea53edf7b7190240e50c2d}
      \strng{authorfullhash}{f170fe32a2ea53edf7b7190240e50c2d}
      \field{sortinit}{7}
      \field{sortinithash}{108d0be1b1bee9773a1173443802c0a3}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{journaltitle}{IEEE Trans. Smart Grid}
      \field{number}{2}
      \field{title}{Resilience enhancement strategy for distribution systems under extreme weather events}
      \field{volume}{9}
      \field{year}{2018}
      \field{pages}{1442\bibrangedash 1451}
      \range{pages}{10}
    \endentry
    \entry{ArifPS18}{article}{}
      \name{author}{6}{}{%
        {{hash=3153205d712a38a4cf21273957a4cd29}{%
           family={Arif},
           familyi={A\bibinitperiod},
           given={Anmar},
           giveni={A\bibinitperiod}}}%
        {{hash=c7dbe653e34afac9875652532c74d5ef}{%
           family={Ma},
           familyi={M\bibinitperiod},
           given={Shanshan},
           giveni={S\bibinitperiod}}}%
        {{hash=e9735f5160a24d53df53e9f5b75435d3}{%
           family={Wang},
           familyi={W\bibinitperiod},
           given={Zhaoyu},
           giveni={Z\bibinitperiod}}}%
        {{hash=735251ff30c1ec1c149b7acd80972b97}{%
           family={Wang},
           familyi={W\bibinitperiod},
           given={Jianhui},
           giveni={J\bibinitperiod}}}%
        {{hash=ca0cc746b080da67d47a90baf1da81ea}{%
           family={Ryan},
           familyi={R\bibinitperiod},
           given={Sarah\bibnamedelima M},
           giveni={S\bibinitperiod\bibinitdelim M\bibinitperiod}}}%
        {{hash=da31f56113cb61a1fef7cf7543eefff8}{%
           family={Chen},
           familyi={C\bibinitperiod},
           given={Chen},
           giveni={C\bibinitperiod}}}%
      }
      \strng{namehash}{f9f498bc36371c5eb11716f324bba09a}
      \strng{fullhash}{15b6e3221a9b723b3d4c549d55c74649}
      \strng{bibnamehash}{f9f498bc36371c5eb11716f324bba09a}
      \strng{authorbibnamehash}{f9f498bc36371c5eb11716f324bba09a}
      \strng{authornamehash}{f9f498bc36371c5eb11716f324bba09a}
      \strng{authorfullhash}{15b6e3221a9b723b3d4c549d55c74649}
      \field{sortinit}{8}
      \field{sortinithash}{a231b008ebf0ecbe0b4d96dcc159445f}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{journaltitle}{IEEE Trans. Power Systems}
      \field{number}{6}
      \field{title}{Optimizing service restoration in distribution systems with uncertain repair time and demand}
      \field{volume}{33}
      \field{year}{2018}
      \field{pages}{6828\bibrangedash 6838}
      \range{pages}{11}
      \verb{doi}
      \verb 10.1109/TPWRS.2018.2855102
      \endverb
    \endentry
    \entry{TanPS18}{article}{}
      \name{author}{4}{}{%
        {{hash=235ef111e422706c92317911d23038a6}{%
           family={Tan},
           familyi={T\bibinitperiod},
           given={Yushi},
           giveni={Y\bibinitperiod}}}%
        {{hash=e85d0d1689753492b43d7a8dd3042184}{%
           family={Das},
           familyi={D\bibinitperiod},
           given={Arindam\bibnamedelima K.},
           giveni={A\bibinitperiod\bibinitdelim K\bibinitperiod}}}%
        {{hash=4887c47b38822f6b699f1e83bf2f1ce0}{%
           family={Arabshahi},
           familyi={A\bibinitperiod},
           given={Payman},
           giveni={P\bibinitperiod}}}%
        {{hash=d2686b705d3d1ed06c68cf5f83c7be8b}{%
           family={Kirschen},
           familyi={K\bibinitperiod},
           given={Daniel\bibnamedelima S.},
           giveni={D\bibinitperiod\bibinitdelim S\bibinitperiod}}}%
      }
      \strng{namehash}{9e9267bca401df1e5e87b7a37d4222d9}
      \strng{fullhash}{9e9267bca401df1e5e87b7a37d4222d9}
      \strng{bibnamehash}{9e9267bca401df1e5e87b7a37d4222d9}
      \strng{authorbibnamehash}{9e9267bca401df1e5e87b7a37d4222d9}
      \strng{authornamehash}{9e9267bca401df1e5e87b7a37d4222d9}
      \strng{authorfullhash}{9e9267bca401df1e5e87b7a37d4222d9}
      \field{sortinit}{9}
      \field{sortinithash}{0a5ebc79d83c96b6579069544c73c7d4}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{journaltitle}{IEEE Trans. Power Systems}
      \field{number}{6}
      \field{title}{Distribution Systems Hardening Against Natural Disasters}
      \field{volume}{33}
      \field{year}{2018}
      \field{pages}{6849\bibrangedash 6860}
      \range{pages}{12}
      \verb{doi}
      \verb 10.1109/TPWRS.2018.2836391
      \endverb
    \endentry
    \entry{TanPS19}{article}{}
      \name{author}{6}{}{%
        {{hash=235ef111e422706c92317911d23038a6}{%
           family={Tan},
           familyi={T\bibinitperiod},
           given={Yushi},
           giveni={Y\bibinitperiod}}}%
        {{hash=bf7939693c0c967ecefa70a6c97db8a6}{%
           family={Qiu},
           familyi={Q\bibinitperiod},
           given={Feng},
           giveni={F\bibinitperiod}}}%
        {{hash=e85d0d1689753492b43d7a8dd3042184}{%
           family={Das},
           familyi={D\bibinitperiod},
           given={Arindam\bibnamedelima K.},
           giveni={A\bibinitperiod\bibinitdelim K\bibinitperiod}}}%
        {{hash=d2686b705d3d1ed06c68cf5f83c7be8b}{%
           family={Kirschen},
           familyi={K\bibinitperiod},
           given={Daniel\bibnamedelima S.},
           giveni={D\bibinitperiod\bibinitdelim S\bibinitperiod}}}%
        {{hash=4887c47b38822f6b699f1e83bf2f1ce0}{%
           family={Arabshahi},
           familyi={A\bibinitperiod},
           given={Payman},
           giveni={P\bibinitperiod}}}%
        {{hash=735251ff30c1ec1c149b7acd80972b97}{%
           family={Wang},
           familyi={W\bibinitperiod},
           given={Jianhui},
           giveni={J\bibinitperiod}}}%
      }
      \strng{namehash}{54ff1c1a0a40ffb323e22ba0d8417280}
      \strng{fullhash}{c799a5ac6913e8788bc89ab3ad44bf09}
      \strng{bibnamehash}{54ff1c1a0a40ffb323e22ba0d8417280}
      \strng{authorbibnamehash}{54ff1c1a0a40ffb323e22ba0d8417280}
      \strng{authornamehash}{54ff1c1a0a40ffb323e22ba0d8417280}
      \strng{authorfullhash}{c799a5ac6913e8788bc89ab3ad44bf09}
      \field{sortinit}{1}
      \field{sortinithash}{4f6aaa89bab872aa0999fec09ff8e98a}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{journaltitle}{IEEE Trans. Power Systems}
      \field{number}{4}
      \field{title}{Scheduling Post-Disaster Repairs in Electricity Distribution Networks}
      \field{volume}{34}
      \field{year}{2019}
      \field{pages}{2611\bibrangedash 2621}
      \range{pages}{11}
      \verb{doi}
      \verb 10.1109/TPWRS.2019.2898966
      \endverb
    \endentry
    \entry{OuyangSS14}{article}{}
      \name{author}{2}{}{%
        {{hash=f44aa511f9d556de294025103d6a7e22}{%
           family={Ouyang},
           familyi={O\bibinitperiod},
           given={Min},
           giveni={M\bibinitperiod}}}%
        {{hash=dc2432f248538e5116dfe6b8a3354518}{%
           family={Dueñas-Osorio},
           familyi={D\bibinithyphendelim O\bibinitperiod},
           given={Leonardo},
           giveni={L\bibinitperiod}}}%
      }
      \strng{namehash}{688132ac35245d342c9ad930767a471c}
      \strng{fullhash}{688132ac35245d342c9ad930767a471c}
      \strng{bibnamehash}{688132ac35245d342c9ad930767a471c}
      \strng{authorbibnamehash}{688132ac35245d342c9ad930767a471c}
      \strng{authornamehash}{688132ac35245d342c9ad930767a471c}
      \strng{authorfullhash}{688132ac35245d342c9ad930767a471c}
      \field{sortinit}{1}
      \field{sortinithash}{4f6aaa89bab872aa0999fec09ff8e98a}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{abstract}{Electric power systems are critical to economic prosperity, national security, public health and safety. However, in hurricane-prone areas, a severe storm may simultaneously cause extensive component failures in a power system and lead to cascading failures within it and across other power-dependent utility systems. Hence, the hurricane resilience of power systems is crucial to ensure their rapid recovery and support the needs of the population in disaster areas. This paper introduces a probabilistic modeling approach for quantifying the hurricane resilience of contemporary electric power systems. This approach includes a hurricane hazard model, component fragility models, a power system performance model, and a system restoration model. These coupled four models enable quantifying hurricane resilience and estimating economic losses. Taking as an example the power system in Harris County, Texas, USA, along with real outage and restoration data after Hurricane Ike in 2008, the proposed resilience assessment model is calibrated and verified. In addition, several dimensions of resilience as well as the effectiveness of alternative strategies for resilience improvement are simulated and analyzed. Results show that among technical, organizational and social dimensions of resilience, the organizational resilience is the highest with a value of 99.964\% (3.445 in a proposed logarithmic scale) while the social resilience is the lowest with a value of 99.760\% (2.620 in the logarithmic scale). Although these values seem high in absolute terms due to the reliability of engineered systems, the consequences of departing from ideal resilience are still high as economic losses can add up to \$83 million per year.}
      \field{issn}{0167-4730}
      \field{journaltitle}{Structural Safety}
      \field{title}{Multi-dimensional hurricane resilience assessment of electric power systems}
      \field{volume}{48}
      \field{year}{2014}
      \field{pages}{15\bibrangedash 24}
      \range{pages}{10}
      \verb{doi}
      \verb https://doi.org/10.1016/j.strusafe.2014.01.001
      \endverb
      \verb{urlraw}
      \verb https://www.sciencedirect.com/science/article/pii/S0167473014000113
      \endverb
      \verb{url}
      \verb https://www.sciencedirect.com/science/article/pii/S0167473014000113
      \endverb
      \keyw{Electric power systems,Resilience,Hurricane hazards,Component fragility,Robustness,Restoration,Networks}
    \endentry
    \entry{CiapessoniENERGIES21}{article}{}
      \name{author}{3}{}{%
        {{hash=798e7e82ea9111936be8bd4f0c66cc09}{%
           family={Ciapessoni},
           familyi={C\bibinitperiod},
           given={Emanuele},
           giveni={E\bibinitperiod}}}%
        {{hash=a82708f55f4fd3aec795fe5dfa35bf3b}{%
           family={Pitto},
           familyi={P\bibinitperiod},
           given={Andrea},
           giveni={A\bibinitperiod}}}%
        {{hash=dccc177c57487b197fe1caa15401bbee}{%
           family={Cirio},
           familyi={C\bibinitperiod},
           given={Diego},
           giveni={D\bibinitperiod}}}%
      }
      \list{publisher}{1}{%
        {MDPI}%
      }
      \strng{namehash}{70a54a0143f84178d9f6a4b01a1b8765}
      \strng{fullhash}{70a54a0143f84178d9f6a4b01a1b8765}
      \strng{bibnamehash}{70a54a0143f84178d9f6a4b01a1b8765}
      \strng{authorbibnamehash}{70a54a0143f84178d9f6a4b01a1b8765}
      \strng{authornamehash}{70a54a0143f84178d9f6a4b01a1b8765}
      \strng{authorfullhash}{70a54a0143f84178d9f6a4b01a1b8765}
      \field{sortinit}{1}
      \field{sortinithash}{4f6aaa89bab872aa0999fec09ff8e98a}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{journaltitle}{Energies}
      \field{number}{16}
      \field{title}{An Application of a Risk-Based Methodology to Anticipate Critical Situations Due to Extreme Weather Events in Transmission and Distribution Grids}
      \field{volume}{14}
      \field{year}{2021}
      \field{pages}{4742}
      \range{pages}{1}
    \endentry
    \entry{PoudelSJ19}{article}{}
      \name{author}{3}{}{%
        {{hash=50615ddf1f2d614efe0d6aa3defda2a8}{%
           family={Poudel},
           familyi={P\bibinitperiod},
           given={S.},
           giveni={S\bibinitperiod}}}%
        {{hash=fdd3ba4e0946fdb005e4296b6ac62b90}{%
           family={Dubey},
           familyi={D\bibinitperiod},
           given={A.},
           giveni={A\bibinitperiod}}}%
        {{hash=2b391f752625d98c4d5a0e23ac07f18d}{%
           family={Bose},
           familyi={B\bibinitperiod},
           given={A.},
           giveni={A\bibinitperiod}}}%
      }
      \strng{namehash}{b0b71ff4a776ab682bb5d904b9d75300}
      \strng{fullhash}{b0b71ff4a776ab682bb5d904b9d75300}
      \strng{bibnamehash}{b0b71ff4a776ab682bb5d904b9d75300}
      \strng{authorbibnamehash}{b0b71ff4a776ab682bb5d904b9d75300}
      \strng{authornamehash}{b0b71ff4a776ab682bb5d904b9d75300}
      \strng{authorfullhash}{b0b71ff4a776ab682bb5d904b9d75300}
      \field{sortinit}{1}
      \field{sortinithash}{4f6aaa89bab872aa0999fec09ff8e98a}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{journaltitle}{IEEE Systems Journal}
      \field{month}{09}
      \field{number}{3}
      \field{title}{Risk-based probabilistic quantification of power distribution system operational resilience}
      \field{volume}{13}
      \field{year}{2020}
      \field{pages}{3506\bibrangedash 3517}
      \range{pages}{12}
    \endentry
    \entry{WeiAM16}{article}{}
      \name{author}{6}{}{%
        {{hash=cdcc947b44bb9b97365a7b9fb939aad6}{%
           family={Wei},
           familyi={W\bibinitperiod},
           given={Yun},
           giveni={Y\bibinitperiod}}}%
        {{hash=0336023bb7f0ba2acdfee3ea6cb9a44a}{%
           family={Ji},
           familyi={J\bibinitperiod},
           given={Chuanyi},
           giveni={C\bibinitperiod}}}%
        {{hash=e5025c89be6c2049ff9a28293dee917a}{%
           family={Galvan},
           familyi={G\bibinitperiod},
           given={Floyd},
           giveni={F\bibinitperiod}}}%
        {{hash=cb97c68fc72de989918f2c449a6a06c4}{%
           family={Couvillon},
           familyi={C\bibinitperiod},
           given={Stephen},
           giveni={S\bibinitperiod}}}%
        {{hash=4f72a9c8aa7cd96dc667d05aa39c3e46}{%
           family={Orellana},
           familyi={O\bibinitperiod},
           given={George},
           giveni={G\bibinitperiod}}}%
        {{hash=d9447007bfffc0051cc5c06104ceb4a8}{%
           family={Momoh},
           familyi={M\bibinitperiod},
           given={James},
           giveni={J\bibinitperiod}}}%
      }
      \strng{namehash}{5c37b89245b9dcb24dde4ac65438e51f}
      \strng{fullhash}{54bcdd0228a5e8a98c3619b9c2696247}
      \strng{bibnamehash}{5c37b89245b9dcb24dde4ac65438e51f}
      \strng{authorbibnamehash}{5c37b89245b9dcb24dde4ac65438e51f}
      \strng{authornamehash}{5c37b89245b9dcb24dde4ac65438e51f}
      \strng{authorfullhash}{54bcdd0228a5e8a98c3619b9c2696247}
      \field{sortinit}{1}
      \field{sortinithash}{4f6aaa89bab872aa0999fec09ff8e98a}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{journaltitle}{Applied Mathematics}
      \field{title}{Non-Stationary Random Process for Large-Scale Failure and Recovery of Power Distribution}
      \field{year}{2016}
    \endentry
    \entry{DavidsonNHR03}{article}{}
      \name{author}{5}{}{%
        {{hash=871dad5a7ff6c0be48c9b611e57bc9d7}{%
           family={Davidson},
           familyi={D\bibinitperiod},
           given={Rachel\bibnamedelima A},
           giveni={R\bibinitperiod\bibinitdelim A\bibinitperiod}}}%
        {{hash=52b746520fa7b938b63c76b72ede92e9}{%
           family={Liu},
           familyi={L\bibinitperiod},
           given={Haibin},
           giveni={H\bibinitperiod}}}%
        {{hash=08293d5afe8b85f454f9d0a604aeb01e}{%
           family={Sarpong},
           familyi={S\bibinitperiod},
           given={Isaac\bibnamedelima K},
           giveni={I\bibinitperiod\bibinitdelim K\bibinitperiod}}}%
        {{hash=b77837d9a2ebca14f616f811cec0ad29}{%
           family={Sparks},
           familyi={S\bibinitperiod},
           given={Peter},
           giveni={P\bibinitperiod}}}%
        {{hash=17b016c4132e11c3a7eb36bfb05b901d}{%
           family={Rosowsky},
           familyi={R\bibinitperiod},
           given={David\bibnamedelima V},
           giveni={D\bibinitperiod\bibinitdelim V\bibinitperiod}}}%
      }
      \list{publisher}{1}{%
        {American Society of Civil Engineers}%
      }
      \strng{namehash}{07c1d6c950d87ae57d0ab6dfb081817b}
      \strng{fullhash}{4c382bf4fc016c899924c487b2de23ae}
      \strng{bibnamehash}{07c1d6c950d87ae57d0ab6dfb081817b}
      \strng{authorbibnamehash}{07c1d6c950d87ae57d0ab6dfb081817b}
      \strng{authornamehash}{07c1d6c950d87ae57d0ab6dfb081817b}
      \strng{authorfullhash}{4c382bf4fc016c899924c487b2de23ae}
      \field{sortinit}{1}
      \field{sortinithash}{4f6aaa89bab872aa0999fec09ff8e98a}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{journaltitle}{Natural Hazards Review}
      \field{number}{1}
      \field{title}{Electric power distribution system performance in {C}arolina hurricanes}
      \field{volume}{4}
      \field{year}{2003}
      \field{pages}{36\bibrangedash 45}
      \range{pages}{10}
    \endentry
    \entry{LiuRESS08}{article}{}
      \name{author}{3}{}{%
        {{hash=52b746520fa7b938b63c76b72ede92e9}{%
           family={Liu},
           familyi={L\bibinitperiod},
           given={Haibin},
           giveni={H\bibinitperiod}}}%
        {{hash=871dad5a7ff6c0be48c9b611e57bc9d7}{%
           family={Davidson},
           familyi={D\bibinitperiod},
           given={Rachel\bibnamedelima A.},
           giveni={R\bibinitperiod\bibinitdelim A\bibinitperiod}}}%
        {{hash=ad137db8ae8b9391ae78d0d14d6180f0}{%
           family={Apanasovich},
           familyi={A\bibinitperiod},
           given={Tatiyana\bibnamedelima V.},
           giveni={T\bibinitperiod\bibinitdelim V\bibinitperiod}}}%
      }
      \strng{namehash}{a98a6938aef2d597ec113ecd6791af5e}
      \strng{fullhash}{a98a6938aef2d597ec113ecd6791af5e}
      \strng{bibnamehash}{a98a6938aef2d597ec113ecd6791af5e}
      \strng{authorbibnamehash}{a98a6938aef2d597ec113ecd6791af5e}
      \strng{authornamehash}{a98a6938aef2d597ec113ecd6791af5e}
      \strng{authorfullhash}{a98a6938aef2d597ec113ecd6791af5e}
      \field{sortinit}{1}
      \field{sortinithash}{4f6aaa89bab872aa0999fec09ff8e98a}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{abstract}{This paper presents new statistical models that predict the number of hurricane- and ice storm-related electric power outages likely to occur in each 3km×3km grid cell in a region. The models are based on a large database of recent outages experienced by three major East Coast power companies in six hurricanes and eight ice storms. A spatial generalized linear mixed modeling (GLMM) approach was used in which spatial correlation is incorporated through random effects. Models were fitted using a composite likelihood approach and the covariance matrix was estimated empirically. A simulation study was conducted to test the model estimation procedure, and model training, validation, and testing were done to select the best models and assess their predictive power. The final hurricane model includes number of protective devices, maximum gust wind speed, hurricane indicator, and company indicator covariates. The final ice storm model includes number of protective devices, ice thickness, and ice storm indicator covariates. The models should be useful for power companies as they plan for future storms. The statistical modeling approach offers a new way to assess the reliability of electric power and other infrastructure systems in extreme events.}
      \field{issn}{0951-8320}
      \field{journaltitle}{Reliability Engineering \& System Safety}
      \field{number}{6}
      \field{title}{Spatial generalized linear mixed models of electric power outages due to hurricanes and ice storms}
      \field{volume}{93}
      \field{year}{2008}
      \field{pages}{897\bibrangedash 912}
      \range{pages}{16}
      \verb{doi}
      \verb https://doi.org/10.1016/j.ress.2007.03.038
      \endverb
      \verb{urlraw}
      \verb https://www.sciencedirect.com/science/article/pii/S0951832007001305
      \endverb
      \verb{url}
      \verb https://www.sciencedirect.com/science/article/pii/S0951832007001305
      \endverb
      \keyw{Spatial,Generalized linear mixed model,Electric power,Hurricane,Ice storm}
    \endentry
    \entry{ReedSJ09}{article}{}
      \name{author}{3}{}{%
        {{hash=220a667df180ea88d7239e9fafeed9b0}{%
           family={Reed},
           familyi={R\bibinitperiod},
           given={Dorothy\bibnamedelima A.},
           giveni={D\bibinitperiod\bibinitdelim A\bibinitperiod}}}%
        {{hash=971e144de6336e389af46d43a8405787}{%
           family={Kapur},
           familyi={K\bibinitperiod},
           given={Kailash\bibnamedelima C.},
           giveni={K\bibinitperiod\bibinitdelim C\bibinitperiod}}}%
        {{hash=09aa1f9c59bc39ccee6b46ca1482d3a4}{%
           family={Christie},
           familyi={C\bibinitperiod},
           given={Richard\bibnamedelima D.},
           giveni={R\bibinitperiod\bibinitdelim D\bibinitperiod}}}%
      }
      \strng{namehash}{8c7a144d4f96212d8ed9c93f658d9809}
      \strng{fullhash}{8c7a144d4f96212d8ed9c93f658d9809}
      \strng{bibnamehash}{8c7a144d4f96212d8ed9c93f658d9809}
      \strng{authorbibnamehash}{8c7a144d4f96212d8ed9c93f658d9809}
      \strng{authornamehash}{8c7a144d4f96212d8ed9c93f658d9809}
      \strng{authorfullhash}{8c7a144d4f96212d8ed9c93f658d9809}
      \field{sortinit}{1}
      \field{sortinithash}{4f6aaa89bab872aa0999fec09ff8e98a}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{journaltitle}{IEEE Systems Journal}
      \field{number}{2}
      \field{title}{Methodology for Assessing the Resilience of Networked Infrastructure}
      \field{volume}{3}
      \field{year}{2009}
      \field{pages}{174\bibrangedash 180}
      \range{pages}{7}
      \verb{doi}
      \verb 10.1109/JSYST.2009.2017396
      \endverb
    \endentry
    \entry{JaechPS19}{article}{}
      \name{author}{4}{}{%
        {{hash=2a8d535be94e11fb3f332f46906e765f}{%
           family={Jaech},
           familyi={J\bibinitperiod},
           given={Aaron},
           giveni={A\bibinitperiod}}}%
        {{hash=c0f73ce79a1970a1bc4fea9c3037d615}{%
           family={Zhang},
           familyi={Z\bibinitperiod},
           given={Baosen},
           giveni={B\bibinitperiod}}}%
        {{hash=d9964335acf3f82957cd1e681220c042}{%
           family={Ostendorf},
           familyi={O\bibinitperiod},
           given={Mari},
           giveni={M\bibinitperiod}}}%
        {{hash=d2686b705d3d1ed06c68cf5f83c7be8b}{%
           family={Kirschen},
           familyi={K\bibinitperiod},
           given={Daniel\bibnamedelima S},
           giveni={D\bibinitperiod\bibinitdelim S\bibinitperiod}}}%
      }
      \list{publisher}{1}{%
        {IEEE}%
      }
      \strng{namehash}{38d12df61a51c4d653561ddf24e16e32}
      \strng{fullhash}{38d12df61a51c4d653561ddf24e16e32}
      \strng{bibnamehash}{38d12df61a51c4d653561ddf24e16e32}
      \strng{authorbibnamehash}{38d12df61a51c4d653561ddf24e16e32}
      \strng{authornamehash}{38d12df61a51c4d653561ddf24e16e32}
      \strng{authorfullhash}{38d12df61a51c4d653561ddf24e16e32}
      \field{sortinit}{1}
      \field{sortinithash}{4f6aaa89bab872aa0999fec09ff8e98a}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{journaltitle}{IEEE Trans. Power Systems}
      \field{number}{1}
      \field{title}{Real-Time Prediction of the Duration of Distribution System Outages}
      \field{volume}{34}
      \field{year}{2018}
      \field{pages}{773\bibrangedash 781}
      \range{pages}{9}
    \endentry
    \entry{CarringtonPS21}{article}{}
      \name{author}{3}{}{%
        {{hash=f1f67a451e1c85a7f8814a0961a413e1}{%
           family={Carrington},
           familyi={C\bibinitperiod},
           given={Nichelle’Le\bibnamedelima K.},
           giveni={N\bibinitperiod\bibinitdelim K\bibinitperiod}}}%
        {{hash=32897e8289ea979b676e887d812472a3}{%
           family={Dobson},
           familyi={D\bibinitperiod},
           given={Ian},
           giveni={I\bibinitperiod}}}%
        {{hash=e9735f5160a24d53df53e9f5b75435d3}{%
           family={Wang},
           familyi={W\bibinitperiod},
           given={Zhaoyu},
           giveni={Z\bibinitperiod}}}%
      }
      \strng{namehash}{5705ede528d8cce4a4b3c3a3d0b1bf13}
      \strng{fullhash}{5705ede528d8cce4a4b3c3a3d0b1bf13}
      \strng{bibnamehash}{5705ede528d8cce4a4b3c3a3d0b1bf13}
      \strng{authorbibnamehash}{5705ede528d8cce4a4b3c3a3d0b1bf13}
      \strng{authornamehash}{5705ede528d8cce4a4b3c3a3d0b1bf13}
      \strng{authorfullhash}{5705ede528d8cce4a4b3c3a3d0b1bf13}
      \field{sortinit}{1}
      \field{sortinithash}{4f6aaa89bab872aa0999fec09ff8e98a}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{journaltitle}{IEEE Trans. Power Systems}
      \field{number}{6}
      \field{title}{Extracting Resilience Metrics From Distribution Utility Data Using Outage and Restore Process Statistics}
      \field{volume}{36}
      \field{year}{2021}
      \field{pages}{5814\bibrangedash 5823}
      \range{pages}{10}
      \verb{doi}
      \verb 10.1109/TPWRS.2021.3074898
      \endverb
    \endentry
    \entry{cerraiACCESS19}{article}{}
      \name{author}{7}{}{%
        {{hash=688dac33314818a1e2fa14dea55c07b4}{%
           family={Cerrai},
           familyi={C\bibinitperiod},
           given={Diego},
           giveni={D\bibinitperiod}}}%
        {{hash=06e3626517601121f3db6d9a2a273ccf}{%
           family={Wanik},
           familyi={W\bibinitperiod},
           given={David\bibnamedelima W},
           giveni={D\bibinitperiod\bibinitdelim W\bibinitperiod}}}%
        {{hash=fc3c328b209af2f54327ca396082659d}{%
           family={Bhuiyan},
           familyi={B\bibinitperiod},
           given={MD\bibnamedelima Abul\bibnamedelima Ehsan},
           giveni={M\bibinitperiod\bibinitdelim A\bibinitperiod\bibinitdelim E\bibinitperiod}}}%
        {{hash=9a6afda6d85ef6012d2c68c3a5eec13b}{%
           family={Zhang},
           familyi={Z\bibinitperiod},
           given={Xinxuan},
           giveni={X\bibinitperiod}}}%
        {{hash=850c9291b236c5e8450327cc620e596f}{%
           family={Yang},
           familyi={Y\bibinitperiod},
           given={Jaemo},
           giveni={J\bibinitperiod}}}%
        {{hash=c7681cc1f3e4b05bca6623168f0b959f}{%
           family={Frediani},
           familyi={F\bibinitperiod},
           given={Maria\bibnamedelima EB},
           giveni={M\bibinitperiod\bibinitdelim E\bibinitperiod}}}%
        {{hash=9f46adae96be4cdff7641f51354a2865}{%
           family={Anagnostou},
           familyi={A\bibinitperiod},
           given={Emmanouil\bibnamedelima N},
           giveni={E\bibinitperiod\bibinitdelim N\bibinitperiod}}}%
      }
      \list{publisher}{1}{%
        {IEEE}%
      }
      \strng{namehash}{00109620b64e86404b23b6aa1deba716}
      \strng{fullhash}{fb5eddb41cacfe4fb12834d0defc3c80}
      \strng{bibnamehash}{00109620b64e86404b23b6aa1deba716}
      \strng{authorbibnamehash}{00109620b64e86404b23b6aa1deba716}
      \strng{authornamehash}{00109620b64e86404b23b6aa1deba716}
      \strng{authorfullhash}{fb5eddb41cacfe4fb12834d0defc3c80}
      \field{sortinit}{2}
      \field{sortinithash}{8b555b3791beccb63322c22f3320aa9a}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{journaltitle}{IEEE Access}
      \field{title}{Predicting storm outages through new representations of weather and vegetation}
      \field{volume}{7}
      \field{year}{2019}
      \field{pages}{29639\bibrangedash 29654}
      \range{pages}{16}
    \endentry
    \entry{kezunovicHICSS18}{inproceedings}{}
      \name{author}{4}{}{%
        {{hash=936dd8c60dea6d3054430a6b1c84bd08}{%
           family={Kezunovic},
           familyi={K\bibinitperiod},
           given={M.},
           giveni={M\bibinitperiod}}}%
        {{hash=db1c8b4a3f2f96412a2d99d1277b29c7}{%
           family={Obradovic},
           familyi={O\bibinitperiod},
           given={Z.},
           giveni={Z\bibinitperiod}}}%
        {{hash=b63c2ca71e17b072b7d3a20957b8a3ea}{%
           family={Djokic},
           familyi={D\bibinitperiod},
           given={T.},
           giveni={T\bibinitperiod}}}%
        {{hash=55a4c73695363bd004069c3a3395561a}{%
           family={Roychoudhury},
           familyi={R\bibinitperiod},
           given={S.},
           giveni={S\bibinitperiod}}}%
      }
      \list{location}{1}{%
        {HI, USA}%
      }
      \strng{namehash}{11e3fcf626c6e5493e646a102d06b62e}
      \strng{fullhash}{11e3fcf626c6e5493e646a102d06b62e}
      \strng{bibnamehash}{11e3fcf626c6e5493e646a102d06b62e}
      \strng{authorbibnamehash}{11e3fcf626c6e5493e646a102d06b62e}
      \strng{authornamehash}{11e3fcf626c6e5493e646a102d06b62e}
      \strng{authorfullhash}{11e3fcf626c6e5493e646a102d06b62e}
      \field{sortinit}{2}
      \field{sortinithash}{8b555b3791beccb63322c22f3320aa9a}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{booktitle}{51st Hawaii Intl. Conf. Syst. Sci.}
      \field{month}{1}
      \field{title}{Systematic framework for integration of weather data into prediction models for the electric grid outage and asset management applications}
      \field{year}{2018}
      \field{pages}{2737\bibrangedash 2746}
      \range{pages}{10}
      \verb{doi}
      \verb 10.24251/HICSS.2018.346
      \endverb
    \endentry
    \entry{DunnNHR18}{article}{}
      \name{author}{5}{}{%
        {{hash=bdc95a16cb7a2cdd71e6d36a99aa93d4}{%
           family={Dunn},
           familyi={D\bibinitperiod},
           given={Sarah},
           giveni={S\bibinitperiod}}}%
        {{hash=e7c01802aa7335ad480d2ef733210e8a}{%
           family={Wilkinson},
           familyi={W\bibinitperiod},
           given={Sean},
           giveni={S\bibinitperiod}}}%
        {{hash=cf4d7abcf34dda355d0f61b6c7096741}{%
           family={Alderson},
           familyi={A\bibinitperiod},
           given={David},
           giveni={D\bibinitperiod}}}%
        {{hash=1ada4f379edc3e248e673f6852dadd52}{%
           family={Fowler},
           familyi={F\bibinitperiod},
           given={Hayley},
           giveni={H\bibinitperiod}}}%
        {{hash=839576978647823eb9ba7ddb8db1d669}{%
           family={Galasso},
           familyi={G\bibinitperiod},
           given={Carmine},
           giveni={C\bibinitperiod}}}%
      }
      \strng{namehash}{703c3f9a714584e59a3d04f123478a5d}
      \strng{fullhash}{c86dc108932c752775ccb216f97aa201}
      \strng{bibnamehash}{703c3f9a714584e59a3d04f123478a5d}
      \strng{authorbibnamehash}{703c3f9a714584e59a3d04f123478a5d}
      \strng{authornamehash}{703c3f9a714584e59a3d04f123478a5d}
      \strng{authorfullhash}{c86dc108932c752775ccb216f97aa201}
      \field{sortinit}{2}
      \field{sortinithash}{8b555b3791beccb63322c22f3320aa9a}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{journaltitle}{Natural Hazards Review}
      \field{number}{1}
      \field{title}{Fragility Curves for Assessing the Resilience of Electricity Networks Constructed from an Extensive Fault Database}
      \field{volume}{19}
      \field{year}{2018}
      \field{pages}{04017019}
      \range{pages}{1}
      \verb{doi}
      \verb 10.1061/(ASCE)NH.1527-6996.0000267
      \endverb
      \verb{eprint}
      \verb https://ascelibrary.org/doi/pdf/10.1061/%28ASCE%29NH.1527-6996.0000267
      \endverb
      \verb{urlraw}
      \verb https://ascelibrary.org/doi/abs/10.1061/%28ASCE%29NH.1527-6996.0000267
      \endverb
      \verb{url}
      \verb https://ascelibrary.org/doi/abs/10.1061/%28ASCE%29NH.1527-6996.0000267
      \endverb
    \endentry
    \entry{ReedWEIA08}{article}{}
      \name{author}{1}{}{%
        {{hash=220a667df180ea88d7239e9fafeed9b0}{%
           family={Reed},
           familyi={R\bibinitperiod},
           given={Dorothy\bibnamedelima A},
           giveni={D\bibinitperiod\bibinitdelim A\bibinitperiod}}}%
      }
      \list{publisher}{1}{%
        {Elsevier}%
      }
      \strng{namehash}{220a667df180ea88d7239e9fafeed9b0}
      \strng{fullhash}{220a667df180ea88d7239e9fafeed9b0}
      \strng{bibnamehash}{220a667df180ea88d7239e9fafeed9b0}
      \strng{authorbibnamehash}{220a667df180ea88d7239e9fafeed9b0}
      \strng{authornamehash}{220a667df180ea88d7239e9fafeed9b0}
      \strng{authorfullhash}{220a667df180ea88d7239e9fafeed9b0}
      \field{sortinit}{2}
      \field{sortinithash}{8b555b3791beccb63322c22f3320aa9a}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{journaltitle}{Journal of Wind Engineering and Industrial Aerodynamics}
      \field{number}{1}
      \field{title}{Electric utility distribution analysis for extreme winds}
      \field{volume}{96}
      \field{year}{2008}
      \field{pages}{123\bibrangedash 140}
      \range{pages}{18}
    \endentry
    \entry{BjarnadoIS13}{article}{}
      \name{author}{3}{}{%
        {{hash=3de4a41e0c0d9b13ba79a1ac8c16916f}{%
           family={Bjarnadottir},
           familyi={B\bibinitperiod},
           given={Sigridur},
           giveni={S\bibinitperiod}}}%
        {{hash=0e1dff109644a2423d611a0bfe123178}{%
           family={Li},
           familyi={L\bibinitperiod},
           given={Yue},
           giveni={Y\bibinitperiod}}}%
        {{hash=fa80c9ec2fb4403a66672441e3d9c15c}{%
           family={Stewart},
           familyi={S\bibinitperiod},
           given={Mark\bibnamedelima G},
           giveni={M\bibinitperiod\bibinitdelim G\bibinitperiod}}}%
      }
      \list{publisher}{1}{%
        {American Society of Civil Engineers}%
      }
      \strng{namehash}{c6eaf1a5392231c2410f70066182ff50}
      \strng{fullhash}{c6eaf1a5392231c2410f70066182ff50}
      \strng{bibnamehash}{c6eaf1a5392231c2410f70066182ff50}
      \strng{authorbibnamehash}{c6eaf1a5392231c2410f70066182ff50}
      \strng{authornamehash}{c6eaf1a5392231c2410f70066182ff50}
      \strng{authorfullhash}{c6eaf1a5392231c2410f70066182ff50}
      \field{sortinit}{2}
      \field{sortinithash}{8b555b3791beccb63322c22f3320aa9a}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{journaltitle}{Journal of Infrastructure Systems}
      \field{number}{1}
      \field{title}{Hurricane risk assessment of power distribution poles considering impacts of a changing climate}
      \field{volume}{19}
      \field{year}{2013}
      \field{pages}{12\bibrangedash 24}
      \range{pages}{13}
    \endentry
    \entry{MurrayPMAPS14}{inproceedings}{}
      \name{author}{2}{}{%
        {{hash=c7d10bce8a34cdc97b2904d4763eee9c}{%
           family={Murray},
           familyi={M\bibinitperiod},
           given={Kirsty},
           giveni={K\bibinitperiod}}}%
        {{hash=897c649d3582e734b8fa86977b61e0a4}{%
           family={Bell},
           familyi={B\bibinitperiod},
           given={Keith\bibnamedelima RW},
           giveni={K\bibinitperiod\bibinitdelim R\bibinitperiod}}}%
      }
      \list{location}{1}{%
        {Durham, United Kingdom}%
      }
      \strng{namehash}{6d0367e371d3264adbf905a1c8de05b7}
      \strng{fullhash}{6d0367e371d3264adbf905a1c8de05b7}
      \strng{bibnamehash}{6d0367e371d3264adbf905a1c8de05b7}
      \strng{authorbibnamehash}{6d0367e371d3264adbf905a1c8de05b7}
      \strng{authornamehash}{6d0367e371d3264adbf905a1c8de05b7}
      \strng{authorfullhash}{6d0367e371d3264adbf905a1c8de05b7}
      \field{sortinit}{2}
      \field{sortinithash}{8b555b3791beccb63322c22f3320aa9a}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{booktitle}{Probabilistic Methods Applied to Power Systems (PMAPS)}
      \field{title}{Wind-related faults on the {GB} transmission network}
      \field{year}{2014}
    \endentry
    \entry{BillintonAllanBook}{book}{}
      \name{author}{2}{}{%
        {{hash=25e121f499e631b1de0e65044e20d40c}{%
           family={Billinton},
           familyi={B\bibinitperiod},
           given={R.},
           giveni={R\bibinitperiod}}}%
        {{hash=2a24807571382479fd122976beb61597}{%
           family={Allan},
           familyi={A\bibinitperiod},
           given={R.\bibnamedelimi N.},
           giveni={R\bibinitperiod\bibinitdelim N\bibinitperiod}}}%
      }
      \list{location}{1}{%
        {New York}%
      }
      \list{publisher}{1}{%
        {Plenum Press}%
      }
      \strng{namehash}{67e100f71ff730b6c9ebf01a0ad66941}
      \strng{fullhash}{67e100f71ff730b6c9ebf01a0ad66941}
      \strng{bibnamehash}{67e100f71ff730b6c9ebf01a0ad66941}
      \strng{authorbibnamehash}{67e100f71ff730b6c9ebf01a0ad66941}
      \strng{authornamehash}{67e100f71ff730b6c9ebf01a0ad66941}
      \strng{authorfullhash}{67e100f71ff730b6c9ebf01a0ad66941}
      \field{sortinit}{2}
      \field{sortinithash}{8b555b3791beccb63322c22f3320aa9a}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{title}{Reliability Evaluation of Power Systems, 2nd ed.}
      \field{year}{1996}
    \endentry
    \entry{BillintonGTD06}{article}{}
      \name{author}{2}{}{%
        {{hash=271a24f5639214eaf38d29e66d2b81bb}{%
           family={Billinton},
           familyi={B\bibinitperiod},
           given={Roy},
           giveni={R\bibinitperiod}}}%
        {{hash=fb84494ae9e81ba81012811a81708634}{%
           family={Singh},
           familyi={S\bibinitperiod},
           given={G.},
           giveni={G\bibinitperiod}}}%
      }
      \strng{namehash}{13f4f09802f77e59664cb8a9a8832845}
      \strng{fullhash}{13f4f09802f77e59664cb8a9a8832845}
      \strng{bibnamehash}{13f4f09802f77e59664cb8a9a8832845}
      \strng{authorbibnamehash}{13f4f09802f77e59664cb8a9a8832845}
      \strng{authornamehash}{13f4f09802f77e59664cb8a9a8832845}
      \strng{authorfullhash}{13f4f09802f77e59664cb8a9a8832845}
      \field{sortinit}{2}
      \field{sortinithash}{8b555b3791beccb63322c22f3320aa9a}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{journaltitle}{IEE Proc.-Gener. Transm. Distrib.}
      \field{number}{1}
      \field{title}{Application of adverse and extreme adverse weather: modeling in transmission and distribution system reliability evaluation}
      \field{volume}{153}
      \field{year}{2006}
    \endentry
    \entry{BrownQuanta09}{report}{}
      \name{author}{1}{}{%
        {{hash=2e7c4d1d38f7906188ea79dadf997b1d}{%
           family={Brown},
           familyi={B\bibinitperiod},
           given={Richard},
           giveni={R\bibinitperiod}}}%
      }
      \list{institution}{1}{%
        {prepared by Quanta Technology for Public Utility Commission of Texas}%
      }
      \strng{namehash}{2e7c4d1d38f7906188ea79dadf997b1d}
      \strng{fullhash}{2e7c4d1d38f7906188ea79dadf997b1d}
      \strng{bibnamehash}{2e7c4d1d38f7906188ea79dadf997b1d}
      \strng{authorbibnamehash}{2e7c4d1d38f7906188ea79dadf997b1d}
      \strng{authornamehash}{2e7c4d1d38f7906188ea79dadf997b1d}
      \strng{authorfullhash}{2e7c4d1d38f7906188ea79dadf997b1d}
      \field{sortinit}{2}
      \field{sortinithash}{8b555b3791beccb63322c22f3320aa9a}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{title}{Cost-Benefit Analysis of the Deployment of Utility Infrastructure Upgrades and Storm Hardening Programs}
      \field{year}{2006}
    \endentry
    \entry{PanteliPS17FLEP}{article}{}
      \name{author}{5}{}{%
        {{hash=b6372e1384c1294d3d990d625dba9b3d}{%
           family={Panteli},
           familyi={P\bibinitperiod},
           given={Mathaios},
           giveni={M\bibinitperiod}}}%
        {{hash=49ba220be7111d9f779704456ec4f52d}{%
           family={Mancarella},
           familyi={M\bibinitperiod},
           given={Pierluigi},
           giveni={P\bibinitperiod}}}%
        {{hash=80a02e1a046d8517a3846788b84c3e42}{%
           family={Trakas},
           familyi={T\bibinitperiod},
           given={Dimitris\bibnamedelima N},
           giveni={D\bibinitperiod\bibinitdelim N\bibinitperiod}}}%
        {{hash=6fc6968af5845b0059a6553072deb576}{%
           family={Kyriakides},
           familyi={K\bibinitperiod},
           given={Elias},
           giveni={E\bibinitperiod}}}%
        {{hash=9f4782e78484c547ef12313746208079}{%
           family={Hatziargyriou},
           familyi={H\bibinitperiod},
           given={Nikos\bibnamedelima D},
           giveni={N\bibinitperiod\bibinitdelim D\bibinitperiod}}}%
      }
      \list{publisher}{1}{%
        {IEEE}%
      }
      \strng{namehash}{419175c6624a85607bf43087d56f0852}
      \strng{fullhash}{881ce63e913df4975e86b8060c72e947}
      \strng{bibnamehash}{419175c6624a85607bf43087d56f0852}
      \strng{authorbibnamehash}{419175c6624a85607bf43087d56f0852}
      \strng{authornamehash}{419175c6624a85607bf43087d56f0852}
      \strng{authorfullhash}{881ce63e913df4975e86b8060c72e947}
      \field{sortinit}{3}
      \field{sortinithash}{ad6fe7482ffbd7b9f99c9e8b5dccd3d7}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{journaltitle}{IEEE Trans. Power Systems}
      \field{number}{6}
      \field{title}{Metrics and quantification of operational and infrastructure resilience in power systems}
      \field{volume}{32}
      \field{year}{2017}
      \field{pages}{4732\bibrangedash 4742}
      \range{pages}{11}
    \endentry
    \entry{RaoufiSUSTAIN20}{article}{}
      \name{author}{3}{}{%
        {{hash=2e5622b1f759de8f1ac7598ec40567e6}{%
           family={Raoufi},
           familyi={R\bibinitperiod},
           given={Habibollah},
           giveni={H\bibinitperiod}}}%
        {{hash=1ceac7932d923c5d20b0340ba70a2828}{%
           family={Vahidinasab},
           familyi={V\bibinitperiod},
           given={Vahid},
           giveni={V\bibinitperiod}}}%
        {{hash=0696522a898cbfc83c5696662939d7db}{%
           family={Mehran},
           familyi={M\bibinitperiod},
           given={Kamyar},
           giveni={K\bibinitperiod}}}%
      }
      \strng{namehash}{d216e5a84ee0f6a238905f1cec5c1eba}
      \strng{fullhash}{d216e5a84ee0f6a238905f1cec5c1eba}
      \strng{bibnamehash}{d216e5a84ee0f6a238905f1cec5c1eba}
      \strng{authorbibnamehash}{d216e5a84ee0f6a238905f1cec5c1eba}
      \strng{authornamehash}{d216e5a84ee0f6a238905f1cec5c1eba}
      \strng{authorfullhash}{d216e5a84ee0f6a238905f1cec5c1eba}
      \field{sortinit}{3}
      \field{sortinithash}{ad6fe7482ffbd7b9f99c9e8b5dccd3d7}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{abstract}{Recently, there has been a focus on natural and man-made disasters with a high-impact low-frequency (HILF) property in electric power systems. A power system must be built with \&ldquo;resilience&rdquo; or the ability to withstand, adapt and recover from disasters. The resilience metrics (RMs) are tools to measure the resilience level of a power system, normally employed for resilience cost\&ndash;benefit in planning and operation. While numerous RMs have been presented in the power system literature; there is still a lack of comprehensive framework regarding the different types of the RMs in the electric power system, and existing frameworks have essential shortcomings. In this paper, after an extensive overview of the literature, a conceptual framework is suggested to identify the key variables, factors and ideas of RMs in power systems and define their relationships. The proposed framework is compared with the existing ones, and existing power system RMs are also allocated to the framework&rsquo;s groups to validate the inclusivity and usefulness of the proposed framework, as a tool for academic and industrial researchers to choose the most appropriate RM in different power system problems and pinpoint the potential need for the future metrics.}
      \field{issn}{2071-1050}
      \field{journaltitle}{Sustainability}
      \field{number}{22}
      \field{title}{Power Systems Resilience Metrics: A Comprehensive Review of Challenges and Outlook}
      \field{volume}{12}
      \field{year}{2020}
      \verb{doi}
      \verb 10.3390/su12229698
      \endverb
      \verb{urlraw}
      \verb https://www.mdpi.com/2071-1050/12/22/9698
      \endverb
      \verb{url}
      \verb https://www.mdpi.com/2071-1050/12/22/9698
      \endverb
    \endentry
    \entry{CarringtonPESGM20}{inproceedings}{}
      \name{author}{4}{}{%
        {{hash=f1f67a451e1c85a7f8814a0961a413e1}{%
           family={Carrington},
           familyi={C\bibinitperiod},
           given={Nichelle’Le\bibnamedelima K.},
           giveni={N\bibinitperiod\bibinitdelim K\bibinitperiod}}}%
        {{hash=c7dbe653e34afac9875652532c74d5ef}{%
           family={Ma},
           familyi={M\bibinitperiod},
           given={Shanshan},
           giveni={S\bibinitperiod}}}%
        {{hash=32897e8289ea979b676e887d812472a3}{%
           family={Dobson},
           familyi={D\bibinitperiod},
           given={Ian},
           giveni={I\bibinitperiod}}}%
        {{hash=e9735f5160a24d53df53e9f5b75435d3}{%
           family={Wang},
           familyi={W\bibinitperiod},
           given={Zhaoyu},
           giveni={Z\bibinitperiod}}}%
      }
      \list{location}{1}{%
        {Montreal, Quebec, Canada}%
      }
      \strng{namehash}{7003ad69cbec41fbb62bb77b143ace90}
      \strng{fullhash}{7003ad69cbec41fbb62bb77b143ace90}
      \strng{bibnamehash}{7003ad69cbec41fbb62bb77b143ace90}
      \strng{authorbibnamehash}{7003ad69cbec41fbb62bb77b143ace90}
      \strng{authornamehash}{7003ad69cbec41fbb62bb77b143ace90}
      \strng{authorfullhash}{7003ad69cbec41fbb62bb77b143ace90}
      \field{sortinit}{3}
      \field{sortinithash}{ad6fe7482ffbd7b9f99c9e8b5dccd3d7}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{booktitle}{IEEE PES General Meeting}
      \field{title}{Extracting resilience statistics from utility data in distribution grids}
      \field{year}{2020}
      \verb{doi}
      \verb 10.1109/PESGM41954.2020.9281596
      \endverb
    \endentry
    \entry{DobsonArxiv23}{article}{}
      \name{author}{1}{}{%
        {{hash=437cde9d873abbd55a4f4a0a0e1da451}{%
           family={Dobson},
           familyi={D\bibinitperiod},
           given={I.},
           giveni={I\bibinitperiod}}}%
      }
      \strng{namehash}{437cde9d873abbd55a4f4a0a0e1da451}
      \strng{fullhash}{437cde9d873abbd55a4f4a0a0e1da451}
      \strng{bibnamehash}{437cde9d873abbd55a4f4a0a0e1da451}
      \strng{authorbibnamehash}{437cde9d873abbd55a4f4a0a0e1da451}
      \strng{authornamehash}{437cde9d873abbd55a4f4a0a0e1da451}
      \strng{authorfullhash}{437cde9d873abbd55a4f4a0a0e1da451}
      \field{sortinit}{3}
      \field{sortinithash}{ad6fe7482ffbd7b9f99c9e8b5dccd3d7}
      \field{labelnamesource}{author}
      \field{labeltitlesource}{title}
      \field{journaltitle}{ArXiv preprint}
      \field{month}{3}
      \field{note}{https://doi.org/10.48550/arXiv.2303.07930}
      \field{title}{Models, metrics, and their formulas for typical electric power system resilience events}
      \field{year}{2023}
      \verb{urlraw}
      \verb https://doi.org/10.48550/arXiv.2303.07930
      \endverb
      \verb{url}
      \verb https://doi.org/10.48550/arXiv.2303.07930
      \endverb
    \endentry
  \enddatalist
\endrefsection
\endinput