A Water Efficiency Dataset for African Data Centers
Extracted source text · arXiv 2412.03716
\documentclass{article}
\usepackage[titletoc]{appendix}
\usepackage[nonatbib,final]{climate_workshop_style}
\usepackage[utf8]{inputenc} % allow utf-8 input
\usepackage[T1]{fontenc} % use 8-bit T1 fonts
\usepackage{hyperref} % hyperlinks
\usepackage{url} % simple URL typesetting
\usepackage{booktabs} % professional-quality tables
\usepackage{amsfonts} % blackboard math symbols
\usepackage{nicefrac} % compact symbols for 1/2, etc.
\usepackage{microtype} % microtypography
\usepackage[table]{xcolor} % color in tables
\usepackage{amsmath} % math environments
\usepackage{multirow} % multiple rows in tables
\usepackage{geometry} % page geometry
\usepackage{graphicx} % include graphics
\usepackage{booktabs,caption}
\usepackage[flushleft]{threeparttable}
\usepackage{booktabs} % For formal tables
% \usepackage{subcaption}
\usepackage{wrapfig,subfig}
\usepackage{cite}
\title{A Water Efficiency Dataset for African Data Centers}
\author{
Noah Shumba \\
Carnegie Mellon University, Africa
\And
Opelo Tshekiso
\\
Carnegie Mellon University, Africa
\And
Pengfei Li
\\
University of California, Riverside
\And
Giulia Fanti
\\
Carnegie Mellon University
\And
Shaolei Ren
\\
University of California, Riverside
}
\begin{document}
\maketitle
\begin{abstract}
AI computing and data centers consume a large amount of freshwater, both directly for cooling
and indirectly for electricity generation. While most attention
has been paid to developed countries such as the U.S.,
this paper presents the first-of-its-kind dataset that
combines nation-level weather and electricity generation data to estimate water usage efficiency
for data centers in 41 African countries across five different climate regions.
We also use our dataset to evaluate and estimate the water consumption of inference on
two large language models (i.e., Llama-3-70B and GPT-4) in 11 selected African countries.
Our findings show that writing
a 10-page report using Llama-3-70B could consume about \textbf{0.7 liters} of water,
while the water consumption by GPT-4 for the same task may go up to about 60 liters. For writing a medium-length email of 120-200 words,
Llama-3-70B and GPT-4 could consume about \textbf{0.13 liters}
and 3 liters of water, respectively.
Interestingly, given the same AI model, 8 out of the 11 selected African countries consume less water than the global average,
mainly because of lower water intensities for electricity generation.
However,
water consumption can be substantially higher
in some African countries with a steppe climate
than the U.S. and global averages, prompting more attention when deploying AI computing in these countries.
Our dataset is publicly available on \href{https://huggingface.co/datasets/masterlion/WaterEfficientDatasetForAfricanCountries/tree/main}{Hugging Face}.
\end{abstract}
\input{introduction}
\input{method}
\input{result}
\input{conclusion}
\section*{References}
\bibliographystyle{ieeetr}
\bibliography{0_main}
\input{appendix}
\end{document}
\section{Introduction}
With the rapid growth of artificial intelligence (AI) and digital services, the demand for data centers has increased substantially~\cite{data_center_energy_growth_2020_McKinsey}. While data center infrastructure
was historically lacking in Africa, the continent's burgeoning digital economy
has recently led to a surge in data center constructions, with a projected market growth of 50\% by 2026 compared to 2021 \cite{DataCenter_Africa_Growth_50_2024}.
Nonetheless, data centers are notorious for their massive energy
usage and water consumption, which have raised significant concerns
even in developed countries such as the U.S. ~\cite{ahmed2014can,DataCenter_US_EnergyUsageReport_2016_LBNL,li2023making}.
More critically, the added pressure on local water resources
is particularly acute in Africa, where many countries are already grappling with extended droughts and water scarcity challenges \cite{unicef,water-africa}.
Therefore, it is important to assess data centers' water consumption in Africa, supporting
healthy development of the data center industry for essential economic growth while
ensuring responsible utilization of limited freshwater resources.
Unfortunately, while recent studies have begun to address
the growing water consumption of data centers and AI computing
\cite{gupta2024dataset,Shaolei_Water_SpatioTemporal_GLB_TCC_2018_7420641,Water_FlexCool_DataCenter_Energy_Water_Tradeoff_AndrewChien_France_eEnergy_2024_10.1145/3632775.3661936,li2023making}, they have predominantly focused
on regions with large data center concentrations, such as the U.S.
and Europe, while leaving out Africa---despite its rapid expansion of data centers and pressing challenges of water scarcity.
In this paper, we address the critical gap in the literature and present a first-of-its-kind water efficiency dataset for data centers in 41 African countries across five distinct climate regions. The dataset includes hourly estimates of water usage efficiency (WUE) for both direct and indirect water consumption over one year. We obtain these estimates by combining weather data from across Africa with the corresponding fuel mix data (i.e., the composition of energy sources in each country).
To demonstrate the utility of this dataset, we consider two recent large language models (LLMs), i.e., Llama-3-70B and GPT-4,
and compare their water consumption for inference in African countries with that in the U.S. and globally. Our results for 11 representative African countries show that writing a 10-page report using Llama-3-70B could consume around 0.7 liters of water while the water consumption by GPT-4 for the same task may go up to nearly 60 liters of water.
Interestingly, our results also suggest that 8 out of the 11 selected African countries (including South Africa and Egypt) have a lower water consumption than the global average for performing the same task.
This is due in part to a less water-intensive fuel mix for electricity generation in these countries. Additionally, some countries such as Morocco are even less water-consuming than the U.S.
Nonetheless, the actual impacts of running AI model inferences on local water resources in these countries are still significant in light
of the high water stress
levels in African countries.
On the other hand, for the same task, some countries such as Namibia are more water-consuming than
the global average. Further compounded by the enduring regional water stress, the higher water consumption may prompt more attention when deploying AI services in these countries.
\section{Background and Methodology}
\label{sec:background}
Our dataset is primarily based on the methodology and modeling of \cite{gupta2024dataset,li2023making}, which study WUE and AI model water consumption with a heavy emphasis on the U.S. data centers.
Like \cite{gupta2024dataset}, we do not model supply chain manufacturing because this aspect often relies on generalized, less accurate data that may not reflect the unique operational practices of individual data centers or computing workloads.
The methodology in \cite{gupta2024dataset} provides
equations for modeling \emph{onsite WUE}, which refers to water directly consumed/evaporated to cool down the facility for each unit of server energy consumption, and \emph{offsite WUE}, which is also
called the electric water intensity factor and refers to indirect water consumption by the generation of electricity that
supplies each unit of data center energy.
Note that water \emph{consumption} is defined as
``water withdrawal minus water discharge'', i.e., the evaporated
portion of water withdrawal that may not be immediately available for reuse
\cite{reig2020guidance}. Data centers commonly consume 80\% of their direct
freshwater withdrawal (in many cases, potable water), while only about 10\% of the water withdrawal is consumed by
typical households and offices \cite{Google_SustainabilityReport_2024}.
\textbf{Onsite WUE.} To assess onsite WUE, \cite{gupta2024dataset} presents an empirical model created from a commercial cooling tower by considering two configurations.
The first configuration is called ``\emph{fixed approach}'', which fixes the differential between wet-bulb and cold water temperatures, and the second one
``\emph{fixed cold water temperature}'' sets a constant cold water temperature.
The WUE formulas for these two configurations are as follows:
\begin{eqnarray}
&\gamma_{\text{Approach}} = \left[-0.0001896 \cdot T_w^2 + 0.03095 \cdot T_w + 0.4442\right]^+,&
\label{eq:1}\\
&\gamma_{\text{ColdWater}} = \left[0.0005112 \cdot T_w^2 - 0.04982 \cdot T_w + 2.387\right]^+,&
\label{eq:2}
\end{eqnarray}
where $T_w$ is the wet-bulb temperature in Fahrenheit and
$\left[x\right]^+=\max\{0,x\}$. Unless otherwise noted, we will focus on Equation~\eqref{eq:2} and simply refer it to as onsite WUE in this paper, because it is typically easier to set a fixed
cold water temperature without adjustment in real systems.
While the onsite WUE for a cooling tower can differ
from other cooling methods such as air economization with water
evaporation, we note that cooling towers are
one of the most commonly adopted and efficient heat rejection mechanisms for data centers \cite{Equinix_EnvironmentalSustainabilityReport_2024,Google_SustainabilityReport_2024}, especially in hot regions like Africa.
\textbf{Offsite WUE.} Electricity generation is water-intensive and must respond to the demand in real time to maintain grid stability.
Thus, similar to
carbon emissions associated with electricity usage,
data centers are also accountable for the electricity water consumption.
Technology companies
including Meta have recently begun to include indirect water consumption for electricity generation in their sustainability reports \cite{Facebook_SustainabilityReport_2024}.
This is critical for holistically understanding the true water impact of data centers, especially in regions where the energy mix includes significant hydroelectric and/or thermal power generation with high water intensities \cite{reig2020guidance,Water_DataCenterFootprint_EnvironmentalResearcHLetters_VT_2021_siddik2021environmental}. Based on \cite{gupta2024dataset,reig2020guidance}, we present the offsite WUE formula as follows:
\begin{equation}
\gamma_{\text{off}}(t) = \frac{\sum_{k}e_{k}(t) \cdot w_{k}}{\sum_{k}e_{k}(t)},
\label{eq:3}
\end{equation}
where $e_k(t)$ is the amount of electricity produced by energy fuel type $k$ (e.g., hydroelectric, geothermal, coal) at time $t$ and $w_k$ is the corresponding water consumption or intensity factor in L/kWh.
\section{Data Collection}
We describe how we collect the necessary data to compute
the onsite and offsite WUEs, respectively.
\begin{wraptable}{r}{0.55\textwidth}
{\footnotesize
\caption{Climatic regions and representative countries}
\label{tab:climatic_regions}
\centering
\begin{tabular}{@{}ll@{}}
\toprule
Climatic Region & Representative Countries \\ \midrule
Rainforest & Republic of the Congo, Gabon, Rwanda \\
Savanna & Morocco, Tunisia \\
Desert & Egypt, Libya \\
Steppe & Namibia, Ethiopia \\
Mediterranean & Algeria, South Africa \\ \bottomrule
\end{tabular}
}
\vspace{-1em}
\end{wraptable}
\textbf{Weather data.}
For weather data, we first identify five distinct climate regions
in Africa: \emph{Rainforest, Savanna, Desert, Steppe, and Mediterranean regions} \cite{africa2024}.
We then collect weather data from the
countries for each climate region, consisting of hourly wet-bulb temperature, humidity, precipitation
over one year
from August 23, 2023 to August 22, 2024.
All the weather data is obtained from WeatherAPI \cite{weatherapi2024}, which is collected via ground-based weather stations and satellite imagery.
We then pick the high and low extremes in terms of the average wet bulb temperature for each region to obtain a representative range.
The selected 11 representative countries are summarized in Table~\ref{tab:climatic_regions}.
\textbf{Energy fuel mix.}
We next collect the energy fuel mix (i.e., the composition of energy fuels) for electricity generation
in each selected country sourced from OurWorldInData \cite{owid-energy-mix}.
Due to the lack of access to fine-grained data, we use
annual granularity for estimating the offsite WUE as done in the prior literature \cite{reig2020guidance}.
Additionally, we need the water intensity of each fuel type in each selected country to compute offsite WUE in \eqref{eq:3}.
While direct data on the water consumption of various energy fuel types for African countries is lacking, \cite{sanchez2020freshwater} studies water withdrawal and consumption throughout different stages of energy production in Africa. Thus,
we use \cite{sanchez2020freshwater} to derive the average water intensity for each energy fuel type in Africa.
\begin{wrapfigure}[15]{r}{0.45\linewidth}
\centering
\includegraphics[width= \linewidth]{figures/monthly_direct_WUE.pdf}
\caption{Average monthly onsite WUE
for desert (red) and rainforest (blue) regions.}
\label{fig:average_onsite_WUE}
\vspace{-0.1in}
\end{wrapfigure}
\section{Dataset Evaluation}
Our final dataset provides onsite and offsite (hourly) WUE for capital cities in 41 African countries.
For clarity, Figure~\ref{fig:average_onsite_WUE} illustrates the monthly averages for a few selected countries in the rainforest and desert regions. The plot clearly illustrates seasonal trends, as well as onsite WUE differences of up to about 40\% between climate regions. Specifically, desert regions generally
are more water-consuming than rainforest regions, which is consistent with the observations in other places \cite{Water_DataCenterEnergy_Tradeoff_Arizona_Real_Measurement_WUE_Monthly_2022_KARIMI2022106194,li2023making}.
Due to space limitations,
we omit the figures for the other regions and offsite WUE while referring
the readers to our dataset for details.
\textbf{Estimating water consumption for AI models.}
To demonstrate the utility of our dataset, we use it to estimate the water consumption of two LLMs, i.e.,
Meta's Llama-3-70B and OpenAI's GPT-4, following the method in \cite{li2023making}.
The tasks we evaluate are to write a comprehensive 10-page report and a medium-length email.
\emph{The details of estimating these AI models' water consumption
and results for writing a medium-length email are available in the appendix.}
Figures~\ref{fig:llama3_70B_long_report} and~\ref{fig:gpt-4_long_report}
indicate that writing a 10-page report using Llama-3-70B and GPT-4 in Africa could consume approximately \textbf{0.7 liters} and \textbf{60 liters} of water, respectively.
In addition, we highlight the following points.
\begin{figure*}[!t]
\centering
\setcounter{subfigure}{0}
\subfloat[S][Llama-3-70B]{
\includegraphics[width=0.42\linewidth]{figures/Llama70B_annotated.pdf}
\label{fig:llama3_70B_long_report}
}%
\subfloat[S][GPT-4]{
\includegraphics[width=0.42\linewidth]{figures/GPT_4_annotated.pdf}
\label{fig:gpt-4_long_report}
}
\caption{Water consumption across 11 selected countries for writing a 10-page report (5,000 tokens) using Llama-3-70B and GPT-4, respectively.}
\label{fig:water_use_by_models_main}
\end{figure*}
First, 8 of the 11 selected African countries have a lower water consumption than the global average. In addition, Morocco and South Africa even have a lower water consumption than the U.S. average. This may be surprising, as Africa is commonly viewed as a water-scarce and dry continent.
The main cause is that these countries
generate electricity from energy fuels
with relatively lower water intensities. Therefore,
although the same AI model consumes more onsite water due to hotter weather, the significantly lower offsite water still makes these countries
less water-consuming overall than the global average.
Nonetheless, the actual impacts of running AI model inferences on local water resources in these countries are still significant in light
of the high water stress
levels in Africa.
Second, our results suggest a correlation between climate conditions with water consumption. Notably, countries categorized under the rainforest region (i.e., Republic of Congo, Rwanda, and Gabon) and the steppe climate (i.e., Ethiopia and Namibia) exhibit higher
or roughly the same
water consumption compared to the global average.
We hypothesize several possible causes.
First, the intrinsic hot and humid conditions of the rainforest climate and the dry, often hot conditions in steppe regions potentially degrade the onsite water efficiency for cooling compared to the global average onsite WUE. This effect is also observed from the onsite WUE
differences among Microsoft's global data center locations. Second, the high offsite water consumption of these countries could suggest that countries in these regions rely more on water-intensive energy fuels like hydroelectric or thermo-electric power.
Indeed, we observe this empirically --- countries with high offsite WUE, such as the Republic of the Congo and Ethiopia rely almost entirely on hydroelectric power.
\begin{wrapfigure}[18]{r}{0.45\linewidth}
\centering
\includegraphics[trim= 0 3cm 0 0, clip, width=\linewidth]{figures/CUE_WUE_compare.pdf}
\caption{Water consumption and (scope-2) carbon emission across various African countries for writing a 10-page report using the Llama-3-70B model.}
\vspace{-0.1in}
\label{fig:water_carbon_llama_3}
\end{wrapfigure}
Third, we show in Figure~\ref{fig:water_carbon_llama_3} the water consumption and (scope-2) carbon emission across various African countries for writing a 10-page report using the Llama-3-70B model.
We see a tradeoff between water consumption and carbon emission, which is consistent with the findings in prior studies \cite{Shaolei_Water_SpatioTemporal_GLB_TCC_2018_7420641}.
This prompts further attention to strike a balance between
water consumption and carbon emission to enable truly sustainable AI in African countries.
Finally, we emphasize the potential uncertainties in our quantitative results. For instance, it is challenging to obtain precise data on the energy fuel mix and the electricity water intensity in Africa.
Moreover, the actual energy consumption of LLM inference may vary depending on the (possibly customized) optimization techniques used by real systems, particularly for the proprietary GPT-4 model. As such, our results should be regarded as first-order estimates rather than precise representations. We encourage AI model developers and data center operators to enhance transparency regarding their most recent water usage, especially in African countries facing water scarcity.
\section{Conclusion}
In this paper, we present the first-of-its-kind dataset
of onsite WUE and offsite WUE for data centers in
41 African countries across five different climate regions.
We also use our dataset to evaluate and estimate the water consumption of inference on
Llama-3-70B and GPT-4 in 11 selected countries.
Our findings underscore the need for region-specific adaptations in
data centers, particularly in cooling systems that can operate efficiently under varying climatic conditions without substantially escalating water usage.
Moreover, the reliance on water-intensive energy sources prompts a broader discussion on the water sustainability practices within the data center industry.
By understanding the water usage efficiency in these different countries, we can make informed decisions that promote sustainable and responsible water use while supporting the growing demand for AI and computing services in Africa.
\section*{Acknowledgement}
Noah Shumba,
Opelo Tshekiso, and
Giulia Fanti acknowledge the support of the Bill \& Melinda Gates Foundation.
Pengfei Li and Shaolei Ren were supported in part by the NSF under
the grant CCF-2324916.
\appendix
\section{Estimating Water Consumption for LLM Inference}\label{appendix:A}
We now present the details of estimating waster consumption
by two LLMs, i.e., Meta's Llama-3-70Angola
B and OpenAI's GPT-4.
We use the following equations to calculate the onsite and offsite water consumption, respectively:
\begin{eqnarray}\nonumber
W_{\text{on}} = \gamma_{\text{on}}\cdot E\;\;\;\;{ \text{and} }\;\;\;\;W_{\text{off}} = \gamma_{\text{off}}\cdot \rho\cdot E,
\label{eq:4}
\end{eqnarray}
where $W$ is the water consumption,
$\gamma$ is the WUE, $\rho$ is the power usage effectiveness (PUE),
$E$ is the server energy consumption for AI models,
and the subscript ``$\text{on}$'' and ``$\text{off}$'' denote
``onsite'' and ``offsite'' wherever applicable, respectively.
Thus, to estimate the LLMs' water consumption, we need
their onsite and offsite WUEs, energy consumption, as well
as the PUE.
\subsection{WUE}
For African countries, we use the average onsite and offsite WUEs from our dataset. For the U.S. and global references,
their average onsite WUEs are obtained from publicly accessible reports
based on Microsoft's U.S. average (0.55 L/kWh) and Equinix's global average (1.07 L/kWh), which
represent efficient hyperscale and colocation
data centers, respectively \cite{equinixPUE,Microsoft_Water_2022_how_azure_cloud}.
Their average offsite WUEs are acquired from the World Resource Institute report \cite{reig2020guidance}.
\subsection{Energy consumption}
The exact LLM inference energy is often lacking in the public domain, especially for those powerful but
proprietary LLMs such as GPT-4 deployed in real-world inference systems.
To estimate LLM inference energy,
some studies resort to a commonly cited claim that
each request of the GPT model family underlying
ChatGPT consumes about 10x the energy as a Google search \cite{Datacenter_EnergyEstimate_IEA_2024}, while others
use GPUs' processing capability in tera operations per second (TOPS) and
the power consumption reported
by manufacturers \cite{Carbon_ChatGPT_Lower_Human_Tomlinson_UCI_ScientificReports_2023_tomlinson2023carbon}. In practice, however, the actual LLM inference energy
consumption depends on a variety of factors, including
the hardware, service level objectives (SLOs), and system optimization \cite{AI_SplitWise_LLM_Inference_ChaojieZhang_Microsoft_ISCA_2024_10609649,AI_DynamoLLM_ResourceManagement_LLM_Inference_energy_UIUC_Microsoft_ChaojieZhang_2024_stojkovic2024dynamollmdesigningllminference}.
To estimate LLM inference energy consumption for a user-facing application (used by, e.g., ChatGPT),
we resort to an online calculator \cite{ecologits-calculator}
and a recent study \cite{AI_DynamoLLM_ResourceManagement_LLM_Inference_energy_UIUC_Microsoft_ChaojieZhang_2024_stojkovic2024dynamollmdesigningllminference}.
These two sources use different methods to calculate
the LLM inference energy consumption, which we describe as follows.
\subsubsection{LLM inference energy estimates by \cite{ecologits-calculator,AI_DynamoLLM_ResourceManagement_LLM_Inference_energy_UIUC_Microsoft_ChaojieZhang_2024_stojkovic2024dynamollmdesigningllminference}}
The online calculator \cite{ecologits-calculator}
estimates the inference energy consumption by various LLMs
based on a transparent methodology.
It first uses the energy measurements from a set of open-sourced models
(mostly on Nvidia A100 GPUs)
to fit an energy consumption curve in terms of the number
of model parameters. For mixture-of-expert model architectures such as the one commonly believed to be used by GPT-4, a range of active model parameters are considered.
The energy measurement takes into account a server's
non-GPU power attributed to the model depending on the fraction
of GPU resources the model utilizes. Nonetheless,
\cite{ecologits-calculator} only considers the token generation phase, while neglecting
the prompt processing phase (i.e., processing user prompts to generate the first output token) which is also energy-intensive \cite{AI_SplitWise_LLM_Inference_ChaojieZhang_Microsoft_ISCA_2024_10609649,AI_PowerCharacterization_LLM_Inference_ChaojieZhang_Microsoft_ASPLOS_2024_10.1145/3620666.3651329,AI_DynamoLLM_ResourceManagement_LLM_Inference_energy_UIUC_Microsoft_ChaojieZhang_2024_stojkovic2024dynamollmdesigningllminference}. In addition, it does not consider batching and
essentially models a lightly-loaded system without request contention.
This can be viewed as a reference system used by industries, e.g., \cite{AI_SplitWise_LLM_Inference_ChaojieZhang_Microsoft_ISCA_2024_10609649,AI_PowerCharacterization_LLM_Inference_ChaojieZhang_Microsoft_ASPLOS_2024_10.1145/3620666.3651329} measure LLM inference energy and power consumption without batching as a reference value for energy and power provisioning,
while \cite{AI_DynamoLLM_ResourceManagement_LLM_Inference_energy_UIUC_Microsoft_ChaojieZhang_2024_stojkovic2024dynamollmdesigningllminference,AI_SplitWise_LLM_Inference_ChaojieZhang_Microsoft_ISCA_2024_10609649} use the latency measurement in such a reference system to set
real SLO targets.
On the other hand, \cite{AI_DynamoLLM_ResourceManagement_LLM_Inference_energy_UIUC_Microsoft_ChaojieZhang_2024_stojkovic2024dynamollmdesigningllminference} measures
the actual GPU energy consumption for LLMs on
enterprise-grade Nvidia DGX H100 servers. Its measurement also considers ``state-of-the-practice'' optimization techniques commonly used in real systems, including batching. Importantly, it considers the prompt processing phase and representative SLO targets, which are both crucial for real-world LLM deployment.
Energy estimates assuming a fully utilized system
without accounting for SLOs may not reflect the industry practice, since a fully-utilized system can lead
to significant SLO violations,
which are not tolerable in real-world LLM
deployment, especially for commercial LLM applications such as
real-time conversations that have strict SLO targets to deliver good quality of experiences \cite{AI_SplitWise_LLM_Inference_ChaojieZhang_Microsoft_ISCA_2024_10609649,AI_DynamoLLM_ResourceManagement_LLM_Inference_energy_UIUC_Microsoft_ChaojieZhang_2024_stojkovic2024dynamollmdesigningllminference}. As a result, server resources for LLM inference are typically provisioned based on the peak demand to ensure SLOs are met at all times.
In other words, the LLM inference servers may not be highly utilized under non-peak loads, resulting in a high energy consumption per request.
For example, the Llama-2-70B inference for a medium-length request on H100 GPUs consumes 9.4 Wh energy without batching \cite{AI_SplitWise_LLM_Inference_ChaojieZhang_Microsoft_ISCA_2024_10609649}, while
the inference energy consumption is still
over 4.0 Wh when batching is applied under various system loads using ``state-of-the-practice'' optimizations (the last column in
Table~II) \cite{AI_DynamoLLM_ResourceManagement_LLM_Inference_energy_UIUC_Microsoft_ChaojieZhang_2024_stojkovic2024dynamollmdesigningllminference}.
The measurement in \cite{AI_DynamoLLM_ResourceManagement_LLM_Inference_energy_UIUC_Microsoft_ChaojieZhang_2024_stojkovic2024dynamollmdesigningllminference} only
includes the GPU energy consumption for a small set of open LLMs.
To account for the non-GPU server energy consumption, we need to multiply the energy consumption in \cite{AI_DynamoLLM_ResourceManagement_LLM_Inference_energy_UIUC_Microsoft_ChaojieZhang_2024_stojkovic2024dynamollmdesigningllminference} by a factor of $1.5\sim2.0$ based on the server power provisioning breakdown \cite{AI_PowerCharacterization_LLM_Inference_ChaojieZhang_Microsoft_ASPLOS_2024_10.1145/3620666.3651329}.
While \cite{ecologits-calculator}
and \cite{AI_DynamoLLM_ResourceManagement_LLM_Inference_energy_UIUC_Microsoft_ChaojieZhang_2024_stojkovic2024dynamollmdesigningllminference} use different methodologies, we note that the server-level inference energy consumption estimated by \cite{ecologits-calculator} (as of November 20, 2024)
is generally lower than that measured by \cite{AI_DynamoLLM_ResourceManagement_LLM_Inference_energy_UIUC_Microsoft_ChaojieZhang_2024_stojkovic2024dynamollmdesigningllminference} for the same model size, assuming the LLM inference system is optimized using
state-of-the-practice techniques in \cite{AI_DynamoLLM_ResourceManagement_LLM_Inference_energy_UIUC_Microsoft_ChaojieZhang_2024_stojkovic2024dynamollmdesigningllminference}.
For example, for Llama-3-70B to write a medium-length email with 250 tokens (or about 120-200 words), \cite{ecologits-calculator} estimates
the inference energy consumption as 2.62 Wh (after removing the PUE of 1.2 for data center overheads), whereas \cite{AI_DynamoLLM_ResourceManagement_LLM_Inference_energy_UIUC_Microsoft_ChaojieZhang_2024_stojkovic2024dynamollmdesigningllminference}
shows the server-level energy consumption is about 10 Wh (after
multiplying the value in the last column of Table~III by 1.6 to account for the non-GPU server energy).
This result might be surprising, as \cite{ecologits-calculator} does not consider system optimization or batching whereas \cite{AI_DynamoLLM_ResourceManagement_LLM_Inference_energy_UIUC_Microsoft_ChaojieZhang_2024_stojkovic2024dynamollmdesigningllminference} uses reasonable ``state-of-the-practice'' optimizations including batching.
Nonetheless, \cite{ecologits-calculator} mostly uses A100 GPUs and does not consider prompt processing energy consumption, whereas
\cite{AI_DynamoLLM_ResourceManagement_LLM_Inference_energy_UIUC_Microsoft_ChaojieZhang_2024_stojkovic2024dynamollmdesigningllminference} uses
H100 GPUs (which may be more energy-consuming than A100 GPUs for LLM inference as shown by \cite{AI_SplitWise_LLM_Inference_ChaojieZhang_Microsoft_ISCA_2024_10609649}) and considers both prompt processing and token generation energy consumption. Additionally, the strict SLOs in real-world deployment prohibits the LLM inference system from being fully utilized. Thus,
the LLM inference energy consumption estimated by \cite{ecologits-calculator} without system optimization could be even lower and still serves as a good reference point.
\subsubsection{Energy consumption for writing a 10-page report and a medium-length email}
For the task of writing a 10-page report,\footnote{The calculator \cite{ecologits-calculator} assumes 5,000 tokens for a 5-page report. Based on the token-to-word ratio \cite{Ref27_OpenAI_Pricing_website}, we consider 5,000 tokens as roughly a 10-page report.} we assume the output is 5,000 tokens and use the estimates by \cite{ecologits-calculator}, since the energy measurement results in \cite{AI_DynamoLLM_ResourceManagement_LLM_Inference_energy_UIUC_Microsoft_ChaojieZhang_2024_stojkovic2024dynamollmdesigningllminference} do not include generating such long outputs using Llama-3-70B.
After removing the PUE of 1.2 for data center overheads,
we estimate that the energy consumption to write a 5,000-token text
by Llama-3-70B and GPT-4 are
52.25 Wh and 4.66 kWh, respectively, based on the results
in \cite{ecologits-calculator} as of November 20, 2024.
Note that, due to the proprietary nature of GPT-4,
\cite{ecologits-calculator} assumes 1,760 billion parameters for GPT-4 with a mixture-of-expert architecture based on the best-known information from various public sources. Additionally,
the energy consumption estimates for models with such large model sizes
are based on extrapolation. As a result, without detailed information from
model owners,
the energy estimates for large proprietary models may have less accuracy than for small/medium open models.
For the task of writing a medium-length email, we assume the output is 250 tokens or about 120-200 words. For Llama-3-70B, by considering
a medium-length prompt and a medium system load,
we estimate
the inference energy consumption as $\sim10$ Wh after
multiplying the value in the last column of Table~III by 1.6 to account for the non-GPU server energy \cite{AI_DynamoLLM_ResourceManagement_LLM_Inference_energy_UIUC_Microsoft_ChaojieZhang_2024_stojkovic2024dynamollmdesigningllminference}.
For GPT-4, we estimate the inference energy consumption as $\sim232$ Wh \cite{ecologits-calculator}.
\subsection{PUE}
PUE is a metric that assesses the energy efficiency of a data center by comparing the total energy consumed by the facility to the energy used by the computing equipment. The ideal PUE is 1.0, indicating 100\% energy efficiency in computing. The inference energy estimate
provided by \cite{ecologits-calculator} assumes a default PUE of 1.2.
The PUE overhead is not needed for calculating the onsite water consumption, but
should be considered when assessing the offsite water consumption.
For different African countries, we consider an average country-/region-wise PUE
%due to the hotter climate and lower data center energy efficiency
provided by
\cite{DataCenter_PUE_Africa_172_2021}. By taking
the lowest when multiple values are presented in \cite{DataCenter_PUE_Africa_172_2021}, the PUE values for the
11 selected African countries
are: 2.3 for Algeria, 2.3 for Egypt, 1.5 for Ethiopia,
1.9 for Gabon, 2.3 for Libya, 2.3 for Morocco, 2.1 for Namibia,
2.0 for
Republic of the Congo,
1.4 for South Africa, 2.3 for Tunisia,
and 1.7 for Rwanda.
We consider Microsoft's U.S. average PUE of 1.17 \cite{Microsoft_Water_2022_how_azure_cloud} and Equinix's
global average PUE of 1.42 \cite{equinixPUE}
for the U.S. and global averages, respectively.
\section{Additional Results}\label{appendix:B}
In addition to the task of writing a 10-page report, we also examine the water consumption of Llama-3-70B and GPT-4 on a more common task: generating a medium-length email, averaging 250 tokens. We show the results in Figure~\ref{fig:water_consumption_email}.
\begin{figure*}[!t]
\centering
\setcounter{subfigure}{0}
\subfloat[S][Llama-3-70B]{
\includegraphics[width=0.42\linewidth]{figures/Llama70B_email_annotated.pdf}
\label{fig:sub_llama3_8B}
}%
\subfloat[S][GPT-4]{
\includegraphics[width=0.42\linewidth]{figures/GPT_4_email_annotated.pdf}
\label{fig:sub_GPT_4}
} \caption{Water consumption across 11 selected countries for writing a medium-length email (250 tokens) using Llama-3-70B and GPT-4, respectively.}
\label{fig:water_consumption_email}
\end{figure*}
\begin{thebibliography}{10}
\bibitem{data_center_energy_growth_2020_McKinsey}
H.~C. Granade, J.~Creyts, A.~Derkach, P.~Farese, S.~Nyquist, and K.~Ostrowski, ``{McKinsey} global energy and materials: Unlocking energy efficiency in the {U.S.} economy,'' Jul. 2009.
\bibitem{DataCenter_Africa_Growth_50_2024}
{Africa Data Centres Association}, ``Data centres in {Africa} focus report 2024.'' \url{https://africadca.org/en/data-centres-in-africa-focus-report-2024}, 2024.
\bibitem{ahmed2014can}
K.~Ahmed, M.~A. Islam, S.~Ren, and G.~Quan, ``Can data center become water $\{$Self-Sufficient$\}$?,'' in {\em 6th Workshop on Power-Aware Computing and Systems (HotPower 14)}, 2014.
\bibitem{DataCenter_US_EnergyUsageReport_2016_LBNL}
A.~Shehabi, S.~J. Smith, N.~Horner, I.~Azevedo, R.~Brown, J.~Koomey, E.~Masanet, D.~Sartor, M.~Herrlin, and W.~Lintner, ``{United States} data center energy usage report,'' {\em Lawrence Berkeley National Laboratory, Berkeley, California. LBNL-1005775}, 2016.
\bibitem{li2023making}
P.~Li, J.~Yang, M.~A. Islam, and S.~Ren, ``Making {AI} less ``thirsty": Uncovering and addressing the secret water footprint of {AI} models,'' {\em Communications of the ACM}, 2024 (accepted).
\bibitem{unicef}
{UNICEF}, ``Water crisis in the horn of africa.'' \url{https://www.unicef.org/documents/water-crisis-horn-africa}.
\newblock (Accessed on 11/25/2024).
\bibitem{water-africa}
{The Water Project}, ``The water crisis: Poverty and water scarcity in africa.'' \url{https://thewaterproject.org/why-water/poverty}.
\newblock (Accessed on 11/25/2024).
\bibitem{gupta2024dataset}
P.~S. Gupta, M.~R. Hossen, P.~Li, S.~Ren, and M.~A. Islam, ``A dataset for research on water sustainability,'' in {\em Proceedings of the 15th ACM International Conference on Future and Sustainable Energy Systems}, pp.~442--446, 2024.
\bibitem{Shaolei_Water_SpatioTemporal_GLB_TCC_2018_7420641}
M.~A. Islam, K.~Ahmed, H.~Xu, N.~H. Tran, G.~Quan, and S.~Ren, ``Exploiting spatio-temporal diversity for water saving in geo-distributed data centers,'' {\em IEEE Transactions on Cloud Computing}, vol.~6, no.~3, pp.~734--746, 2018.
\bibitem{Water_FlexCool_DataCenter_Energy_Water_Tradeoff_AndrewChien_France_eEnergy_2024_10.1145/3632775.3661936}
W.~E. Gnibga, A.~A. Chien, A.~Blavette, and A.~C. Orgerie, ``Flexcooldc: Datacenter cooling flexibility for harmonizing water, energy, carbon, and cost trade-offs,'' in {\em Proceedings of the 15th ACM International Conference on Future and Sustainable Energy Systems}, e-Energy '24, (New York, NY, USA), p.~108–122, Association for Computing Machinery, 2024.
\bibitem{reig2020guidance}
P.~Reig, T.~Luo, E.~Christensen, and J.~Sinistore, ``Guidance for calculating water use embedded in purchased electricity,'' {\em World Resources Institute}, 2020.
\bibitem{Google_SustainabilityReport_2024}
Google, ``Environmental report.'' \url{https://sustainability.google/reports/}, 2024.
\bibitem{Equinix_EnvironmentalSustainabilityReport_2024}
Equinix, ``Sustainability report.'' \url{https://sustainability.equinix.com/wp-content/uploads/2024/07/Equinix-Inc_2023-Sustainability-Report.pdf}, 2024.
\bibitem{Facebook_SustainabilityReport_2024}
Meta, ``Sustainability report.'' \url{https://sustainability.atmeta.com/2024-sustainability-report/}, 2024.
\bibitem{Water_DataCenterFootprint_EnvironmentalResearcHLetters_VT_2021_siddik2021environmental}
M.~A.~B. Siddik, A.~Shehabi, and L.~Marston, ``The environmental footprint of data centers in the {United States},'' {\em Environmental Research Letters}, vol.~16, no.~6, p.~064017, 2021.
\bibitem{africa2024}
A.~Kröner, J.~I. Clarke, R.~W. Steel, D.~N. McMaster, R.~K. Gardiner, K.~B. Dickson, A.~L. Mabogunje, A.~Smedley, J.~F. Middleton, and D.~S. Nicol, ``Africa.'' Encyclopedia Britannica, Aug 2024.
\newblock Accessed: 30 August 2024.
\bibitem{weatherapi2024}
``{WeatherAPI}.'' \url{https://www.weatherapi.com/}.
\newblock Accessed: 30 August 2024.
\bibitem{owid-energy-mix}
H.~Ritchie and P.~Rosado, ``Energy mix,'' {\em Our World in Data}, 2020.
\newblock https://ourworldindata.org/energy-mix.
\bibitem{sanchez2020freshwater}
R.~G. Sanchez, R.~Seliger, F.~Fahl, L.~De~Felice, T.~B. Ouarda, and F.~Farinosi, ``Freshwater use of the energy sector in africa,'' {\em Applied Energy}, vol.~270, p.~115171, 2020.
\bibitem{Water_DataCenterEnergy_Tradeoff_Arizona_Real_Measurement_WUE_Monthly_2022_KARIMI2022106194}
L.~Karimi, L.~Yacuel, J.~Degraft-Johnson, J.~Ashby, M.~Green, M.~Renner, A.~Bergman, R.~Norwood, and K.~L. Hickenbottom, ``Water-energy tradeoffs in data centers: A case study in hot-arid climates,'' {\em Resources, Conservation and Recycling}, vol.~181, p.~106194, 2022.
\bibitem{equinixPUE}
``2023 equinix sustainability report.'' \url{https://sustainability.equinix.com/wp-content/uploads/2024/07/Equinix-Inc_2023-Sustainability-Report.pdf}, 2023.
\newblock [Accessed 29-08-2024].
\bibitem{Microsoft_Water_2022_how_azure_cloud}
N.~Walsh, ``How {Microsoft} measures datacenter water and energy use to improve {Azure Cloud} sustainability,'' {\em Microsoft Azure Blog}, April 2022.
\bibitem{Datacenter_EnergyEstimate_IEA_2024}
{International Energy Agency}, ``Electricity 2024.'' \url{https://www.iea.org/reports/electricity-2024}, 2024.
\bibitem{Carbon_ChatGPT_Lower_Human_Tomlinson_UCI_ScientificReports_2023_tomlinson2023carbon}
B.~Tomlinson, R.~W. Black, D.~J. Patterson, and A.~W. Torrance, ``The carbon emissions of writing and illustrating are lower for {AI} than for humans,'' {\em Scientific Reports}, vol.~14, February 2024.
\bibitem{AI_SplitWise_LLM_Inference_ChaojieZhang_Microsoft_ISCA_2024_10609649}
P.~Patel, E.~Choukse, C.~Zhang, A.~Shah, I.~Goiri, S.~Maleki, and R.~Bianchini, ``Splitwise: Efficient generative {LLM} inference using phase splitting,'' in {\em 2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA)}, pp.~118--132, 2024.
\bibitem{AI_DynamoLLM_ResourceManagement_LLM_Inference_energy_UIUC_Microsoft_ChaojieZhang_2024_stojkovic2024dynamollmdesigningllminference}
J.~Stojkovic, C.~Zhang, I.~Goiri, J.~Torrellas, and E.~Choukse, ``{DynamoLLM}: Designing {LLM} inference clusters for performance and energy efficiency,'' in {\em IEEE International Symposium on High-Performance Computer Architecture ({HPCA})}, 2025.
\bibitem{ecologits-calculator}
A.~B. Samuel~Rincé and V.~Defour, ``Ecologits calculator.'' \url{https://huggingface.co/spaces/genai-impact/ecologits-calculator}, 2024.
\bibitem{AI_PowerCharacterization_LLM_Inference_ChaojieZhang_Microsoft_ASPLOS_2024_10.1145/3620666.3651329}
P.~Patel, E.~Choukse, C.~Zhang, I.~n. Goiri, B.~Warrier, N.~Mahalingam, and R.~Bianchini, ``Characterizing power management opportunities for llms in the cloud,'' in {\em Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3}, ASPLOS '24, (New York, NY, USA), p.~207–222, Association for Computing Machinery, 2024.
\bibitem{Ref27_OpenAI_Pricing_website}
{OpenAI}, ``{OpenAI API Pricing}.'' \url{https://openai.com/api/pricing/}.
\bibitem{DataCenter_PUE_Africa_172_2021}
{Africa Data Centres Association}, ``African climate \& datacenter {PUE} 2021.'' \url{https://africadca.org/wp-content/uploads/2022/05/PUE-Climate-Hydrogen-in-African-DCs-White-paper.pdf}, 2021.
\end{thebibliography}