A note on numerical evaluation of conditional Akaike information for nonlinear mixed-effects models
Extracted source text · arXiv 2411.14185
\documentclass[12pt]{article}
\usepackage{amsmath}
\usepackage{graphicx}
\usepackage{enumerate}
\usepackage{natbib}
\usepackage{url} % not crucial - just used below for the URL
\usepackage{placeins}
\usepackage{booktabs}
\usepackage{lineno}
\usepackage[title]{appendix}
\usepackage{pdfpages}
\usepackage[normalem]{ulem}
\usepackage{xcolor}
\newtheorem{definition}{Definition}
\newtheorem{lemma}{Lemma}
\newtheorem{theorem}{Theorem}
%\pdfminorversion=4
% NOTE: To produce blinded version, replace "0" with "1" below.
\newcommand{\blind}{1}
% DON'T change margins - should be 1 inch all around.
\addtolength{\oddsidemargin}{-.5in}%
\addtolength{\evensidemargin}{-1in}%
\addtolength{\textwidth}{1in}%
\addtolength{\textheight}{1.7in}%
\addtolength{\topmargin}{-1in}%
\allowdisplaybreaks
\begin{document}
%\bibliographystyle{natbib}
\def\spacingset#1{\renewcommand{\baselinestretch}%
{#1}\small\normalsize} \spacingset{1}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\if1\blind
{
\title{\bf A note on numerical evaluation of conditional Akaike information for nonlinear mixed-effects models}
\author{Nan Zheng\\
Department of Mathematics and Statistics,\\ Memorial University of Newfoundland\\
and \\
Noel Cadigan\thanks{Email: Noel.Cadigan@mi.mun.ca
}\hspace{.2cm} \\
Centre for Fisheries Ecosystems Research,\\ Fisheries and Marine Institute of Memorial University of Newfoundland\\
and \\
James T. Thorson\\
Resource Ecology and Fisheries
Management,\\ Alaska Fisheries Science
Center,\\ National Marine Fisheries Service,\\
National Oceanic and Atmospheric
Administration
}
\maketitle
} \fi
\if0\blind
{
\bigskip
\bigskip
\bigskip
\begin{center}
{\LARGE\bf A note on numerical evaluation of conditional Akaike information for nonlinear mixed-effects models}
\end{center}
\medskip
} \fi
\bigskip
\begin{abstract}
We propose two methods to evaluate the conditional Akaike information (cAI) for nonlinear mixed-effects models with no restriction on cluster size. Method 1 is designed for continuous data and includes formulae for the derivatives of fixed and random effects estimators with respect to observations. Method 2, compatible with any type of observation, requires modeling the marginal (or prior) distribution of random effects as a multivariate normal distribution. Simulations show that Method 1 performs well with Gaussian data but struggles with skewed continuous distributions, whereas Method 2 consistently performs well across various distributions, including normal, gamma, negative binomial, and Tweedie, with flexible link functions. Based on our findings, we recommend Method 2 as a distributionally robust cAI criterion for model selection in nonlinear mixed-effects models.
%The text of your abstract. 200 or fewer words.
\end{abstract}
\noindent%
{\it Keywords:} Nonlinear mixed-effects model; Model selection; Marginal likelihood; Conditional likelihood; Maximum marginal likelihood estimator; Empirical Bayes estimator %3 to 6 keywords, that do not appear in the title
\vfill
\newpage
%\spacingset{1.9} % DON'T change the spacing!
\section{Introduction}
\label{sec:intro}
Nonlinear mixed-effects models (NMMs) are widely used across disciplines such as medicine, public health, pharmacology, fisheries, and ecology, where the quantities of interest are nonlinear functions of both fixed and random effects \citep{lindstrom1990nonlinear}. In these models, the log-joint likelihood of the data ($y$) and random effects (REs, $\Psi$) conditional on the fixed effects (parameters, $\theta$) can be expressed as $\log{ f(y,\Psi|\theta)} = \log{ f(y|\theta,\Psi)} + \log{ f(\Psi|\theta)}$. Maximum likelihood estimation of $\theta$ is achieved by optimizing the log-marginal likelihood, $\log{ f(y|\theta)}$, which results from integrating out the REs from the joint density.
Practitioners of NMMs often need to select the optimal model from a wide range of alternatives \citep[e.g.,][]{cadigan2022complex}, with many relying on the conventional Akaike information criterion \citep[AIC,][]{Akaike1973AIC}. AIC is intended to estimate the expected log-marginal likelihood for out-of-sample predictions, where the degree of freedom correction adjusts for the expected difference between in-sample and out-of-sample prediction \citep[termed ``model optimism" in][]{hooten2015guide}. However, \cite{vaida2005conditional} pointed out that when the REs are of primary interest, rather than merely a device for modeling the correlation among responses, the Akaike Information (AI) and the corresponding model selection criterion should be redefined using the log-conditional likelihood, $\log{ f(y|\hat{\theta}, \hat{\Psi})}$. These are termed conditional AI (cAI) and conditional AIC (cAIC), respectively. Here and throughout the paper, the hat notation denotes estimators of parameters or predictors of REs. Studies also suggest that the conventional AIC is too conservative for mixed-effects model selection, while the cAIC performs more reliably in this context \citep[e.g.][]{yu2012conditional,wood2016smoothing}.
Given the strong relevance of cAIC to mixed-effects model applications, its formulation and evaluation have been extensively studied across various settings, including linear mixed-effects models with independent Gaussian observations of known variance \citep{liang2008note}, generalized linear mixed models with known dispersion \citep{yu2012conditional}, and smooth models \citep{wood2016smoothing}, among others. However, the lack of a unified cAIC evaluation methodology makes it difficult for practitioners of NMMs to identify if there are appropriate cAIC formulae in the literature that correspond to their specific research settings. This challenge is further compounded by the development of advanced modeling tools like TMB \citep{kristensen2016tmb}, which enable the exploration of NMMs that do not easily fit into the existing cAIC frameworks. On the other hand, some existing cAIC methods may be further generalized and improved using these advanced modeling tools. For instance, \cite{liang2008note} suggests evaluating their cAIC using numerical approximations based on small perturbations of the observed data, which can be challenging to implement in practice. Leveraging R \citep{r2018citation} and TMB's strong automatic differentiation (up to order three) capabilities, we propose a simpler method in Sec. \ref{sec:meth}, applicable to broader scenarios including correlated observations, not addressed by \cite{liang2008note}. This example also highlights our aim: to explore cAIC evaluation using recent advances in numerical methods and corresponding inferential theory for NMMs \citep{zheng2021frequentist,Zheng2021P2}.
The rest of the paper is organized as follows. In Sec. \ref{sec:meth}, we introduce two cAIC formulas. The first formula, suited for continuous observations, requires evaluation using provided derivatives of the parameter and RE estimators with respect to the data. The second formula, which accommodates all observation types, assumes that the REs are modeled with a multivariate normal distribution. In Sec. \ref{sec:simulation_October_21_2024}, we perform extensive simulations to test both cAIC formulas across different data types. Concluding remarks are presented in Sec. \ref{sec:conc}.
\section{Methods}
\label{sec:meth}
\subsection{Notation and conditional AIC}\label{notation_October_8_2024}
Let $y, \theta$ and $\Psi$ represent the vectors of all observations, fixed effects (parameters) and REs, respectively. The log-joint likelihood is denoted as
\begin{linenomath*}
\begin{align}\label{EQ:log_joint_likelihood_October_8_2024}
\begin{split}
l_j(y,\theta,\Psi) &= \log{ f(y,\Psi|\theta)} = \log{ f(y|\theta,\Psi)} + \log{ f(\Psi|\theta)} = l_c(\theta,\Psi|y) + l_r(\theta,\Psi).
\end{split}
\end{align}
\end{linenomath*}
Assume that the true distribution of $y$ is $g(y|u)$, where $u$ is the true REs vector with distribution $p(u)$. The vector $u$ may differ in length from $\Psi$, but we assume that $g(y|u)=f(y|\theta_o,\Psi_o)$ for some $\Psi_o$ and $\theta_o$. This assumption is a general requirement, either explicitly or implicitly, in various derivations of cAIC \citep[see, e.g.,][]{vaida2005conditional,liang2008note}. Notably, $\theta_o$ and $\Psi_o$ may have different dimensions from $\theta$ and $\Psi$, respectively. For instance, if $u$ is longer than $\Psi$, then $\Psi_o$ may match the length of $u$, while $\Psi$ assumes that some components of $\Psi_o$ are identical. Let $\hat{\theta}$ denote the maximum marginal likelihood estimator (MMLE) of $\theta$, and $\hat{\Psi}(y)$ the empirical Bayes estimator of $\Psi$ \citep{kass1989approximate}. \cite{vaida2005conditional} defined the cAI as follows.
\begin{definition}
The conditional Akaike information is defined to be
\begin{linenomath*}
\begin{align}\label{EQ:cAI_October_9_2024}
\begin{split}
\mathrm{cAI} &= -2 \,\mathrm{E}_{g(y,u)}\left( \mathrm{E}_{g(y^*|u)}\left[ \log f\{ y^*|\hat{\theta}(y), \hat{\Psi}(y) \} \right] \right) \\
&= -2\int \log f\{ y^*|\hat{\theta}(y), \hat{\Psi}(y) \}\, g(y^*|u) g(y|u)p(u) dy^*dydu,
\end{split}
\end{align}
\end{linenomath*}
where $y^*$ is the prediction dataset which is independent of $y$ conditional on $u$, and from the same distribution $g(\cdot|u)$ as $y$.
\end{definition}
Given the assumption $g(y|u) = f(y|\theta_o, \Psi_o)$, the conditional expectations with respect to $y^*$ and $y$ can be computed. However, since $p(u)$ is unknown, the derivation of cAIC should avoid evaluating the expectation over $u$.
Following \cite{vaida2005conditional} and \cite{liang2008note}, we define a cluster as a set of observations that share the same RE component. Let $T$ denote the total number of clusters, and $n_t$ the number of observations in the $t^{\mathrm{th}}$ cluster.
\subsection{Evaluation of conditional AIC}\label{evaluation_cAIC_October_9_2024}
The cAIC formula presented in \cite{liang2008note} includes the derivative of the mean estimators of $y$ with respect to $y$, which can be expressed as the derivatives of $\hat{\theta}(y)$ and $\hat{\Psi}(y)$ with respect to $y$. These derivatives are provided in the following lemma.
\begin{lemma}\label{lemma:derivatives_hattheta_Psi_wrt_y_October_10_2024}
Assume that the log-marginal likelihood $l(y,\theta)$ and log-joint likelihood $l_j(y,\theta,\Psi)$ have well-defined second-order derivatives. For the maximum marginal likelihood estimator $\hat{\theta}(y)$, obtained by maximizing $l(y,\theta)$ with respect to $\theta$, and the empirical Bayes estimator $\hat{\Psi}(y)$, obtained by maximizing $l_j(y,\hat{\theta},\Psi)$ with respect to $\Psi$, the first-order derivatives with respect to $y$ are given by
\begin{linenomath*}
\begin{align}\label{EQ:derivatives_hattheta_Psi_wrt_y_October_10_2024}
\begin{split}
\dfrac{d\hat{\theta}^{\top}(y)}{d y} &= -\dfrac{\partial^2 l(y,\hat{\theta})}{\partial y\partial\hat{\theta}^{\top}}\left\lbrace \dfrac{\partial^2 l(y,\hat{\theta})}{\partial\hat{\theta}\partial\hat{\theta}^{\top}} \right\rbrace^{-1},\\
\dfrac{d\hat{\Psi}^{\top}(y)}{d y} &= -\left\lbrace \dfrac{\partial^2 l_j(y,\hat{\theta},\hat{\Psi})}{\partial y\partial\hat{\Psi}^{\top}} + \dfrac{d\hat{\theta}^{\top}(y)}{d y}\dfrac{\partial^2 l_j(y,\hat{\theta},\hat{\Psi})}{\partial \hat{\theta}\partial\hat{\Psi}^{\top}} \right\rbrace \left\lbrace \dfrac{\partial^2 l_j(y,\hat{\theta},\hat{\Psi})}{\partial \hat{\Psi}\partial\hat{\Psi}^{\top}} \right\rbrace^{-1}.
\end{split}
\end{align}
\end{linenomath*}
\end{lemma}
The derivation of Lemma \ref{lemma:derivatives_hattheta_Psi_wrt_y_October_10_2024} is given in Appendix \ref{app:derivatives_derivation_October_30_2024}. These derivatives can be efficiently computed using TMB and the ``numericDeriv()" function in R, leveraging their strong numerical differentiation capabilities. For instance, by treating $y$ as parameters rather than data, TMB can quickly compute $\partial^2 l_j(y,\hat{\theta},\hat{\Psi})/\partial y\partial\hat{\Psi}^{\top}$ using automatic differentiation. Let $\Theta=(\theta^{\top},\Psi^{\top})^{\top}$. These derivatives enable the evaluation of the first-order Taylor expansion of the estimator $\hat{\Theta}(y)$ with respect to $y$, thereby supporting the following theorem for calculating cAIC for continuous data. The proof of this theorem can be found in Appendix \ref{app:derivation_4_October_30_2024}. Let $\mathrm{COV}_{g(y|u)}(y)$ denote the conditional covariance of $y$ given the true REs $u$.
\begin{theorem}\label{theorem_cAIC_continous_October_15_2024}
Assume that the data $y$ follow the true density $g(y|u)=f(y|\theta_o,\Psi_o)$ for some $\theta_o$ and some REs $\Psi_o$ and $u$ with distribution $p(u)$. Let the data be modeled by a continuous density denoted by $f(y|\theta,\Psi)$. Then to the first order of Taylor expansion, cAI in (\ref{EQ:cAI_October_9_2024}) can be estimated by
\begin{linenomath*}
\begin{align}\label{EQ:CAIC_October_15_2024}
\begin{split}
\mathrm{cAIC} = &-2\, \log f\{y|\hat{\theta}(y),\hat{\Psi}(y)\} \\
&+ 2\, \mathrm{trace}\left\lbrace \dfrac{\partial\hat{\Theta}^{\top}(y)}{\partial y}\left. \dfrac{\partial^2 \log f(y|\Theta_o)}{\partial\Theta_o\partial y^{\top}}\right|_{y=\mu_y}\mathrm{COV}_{g(y|u)}(y) \right\rbrace,
\end{split}
\end{align}
\end{linenomath*}
where $\mu_y=\mathrm{E}_{g(y|u)}(y)$.
\end{theorem}
Note that the modeling distribution of $\Psi$ is not required for Theorem \ref{theorem_cAIC_continous_October_15_2024}. When the data are independent, normally distributed with known variance, with an identity link function, Eq. (\ref{EQ:CAIC_October_15_2024}) is exact and matches Eq. (4) in \cite{liang2008note}. Eq. (\ref{EQ:CAIC_October_15_2024}) also applies to both Gaussian and non-Gaussian continuous data with unknown correlation structures, and the second-order derivative of $\log f(y|\Theta_o)$ can be efficiently evaluated using the same method as for Lemma \ref{lemma:derivatives_hattheta_Psi_wrt_y_October_10_2024}. However, the approximation order depends on several factors, such as the link function, as well as the skewness and kurtosis of the observational distribution, and thus cannot be universally defined. In Sec. \ref{sec:gamma_October_21_2024}, we will show that for skewed distributions, such as the gamma distribution, and when a non-identity link function is used, the performance of Eq. (\ref{EQ:CAIC_October_15_2024}) may degrade. Additionally, we aim to accommodate discrete distributions. Therefore, in Appendix \ref{app:derivation_6_October_30_2024}, we derived the Taylor expansion of cAI with respect to $\hat{\Theta}$ around $\Theta_o$, yielding the following expressions for cAIC:
\begin{linenomath*}
\begin{align}\label{EQ:CAIC_exact_October_17_2024}
\begin{split}
\mathrm{cAIC} = &-2\, \log f\{y|\hat{\theta}(y),\hat{\Psi}(y)\} + 2\,p_c \\
&+ 2\, \mathrm{E}_{g(y,u)}\left[\left\lbrace \hat{\Psi}(y)-\Psi_o\right\rbrace ^{\top}\,\dfrac{\partial\log f(y|\theta_o,\Psi_o)}{\partial\Psi_o} \right]\\
= &-2\, \log f\{y|\hat{\theta}(y),\hat{\Psi}(y)\} + 2\,p_c + 2\,q \\
&+ 2\,\mathrm{trace}\left( \mathrm{E}_{p(u)}\left[\dfrac{\partial}{\partial\Psi_o^{\top}}\mathrm{E}_{g(y|u)}\{\hat{\Psi}(y)-\Psi_o\} \right] \right),
\end{split}
\end{align}
\end{linenomath*}
where $p_c$ is the number of parameters in the log-conditional likelihood $\log f(y|\theta,\Psi)$, and $q$ is the number of REs. Evaluating the last term requires a distributional assumption about the REs. We assume the RE model, $f(\Psi|\theta)$ (not the true RE distribution $p(u)$), is a multivariate normal distribution (MVN), which is commonly used for modeling REs. Let the double dots in $\ddot{l}_j^{-1}(y,\theta,\Psi)$ and $\ddot{l}_r(\theta,\Psi)$ represent second-order derivatives with respect to $\Psi$. In Appendix \ref{app:derivation_6_October_30_2024}, we proved the following theorem for cAIC.
\begin{theorem}\label{theorem_cAIC_October_18_2024}
Assume that the data $y$ follow the true density $g(y|u)=f(y|\theta_o,\Psi_o)$ for some $\theta_o$ and some REs $\Psi_o$ and $u$ with distribution $p(u)$. Let the REs be modeled as a MVN. Then, the cAIC can be estimated as:
\begin{linenomath*}
\begin{align}\label{EQ:CAIC_October_18_2024}
\begin{split}
\mathrm{cAIC} = &-2\, \log f\{y|\hat{\theta}(y),\hat{\Psi}(y)\} + 2\,p_c + 2\,q \\
&- 2\,\mathrm{trace}\left\{ \ddot{l}_j^{-1}(y,\hat{\theta},\hat{\Psi})\,\ddot{l}_r(\hat{\theta},\hat{\Psi}) \right\} + O(1/n_t).
\end{split}
\end{align}
\end{linenomath*}
\end{theorem}
The last two terms of Eq. (\ref{EQ:CAIC_October_18_2024}), namely, the trace term and the $O(1/n_t)$ term, both arise from the trace component of Eq. (\ref{EQ:CAIC_exact_October_17_2024}).
If the data are uninformative about $\Psi$ then the trace term will be $\approx q$ and $\mathrm{cAIC} \approx -2\, \log f\{y|\hat{\theta}(y),\hat{\Psi}(y)\} + 2\,p_c$. On the other hand, if the data are highly informative about the $\Psi$ then the trace term will be close to zero and $\mathrm{cAIC} \approx -2\, \log f\{y|\hat{\theta}(y),\hat{\Psi}(y)\} + 2\,(p_c + q)$. Appendix \ref{app:derivation_6_October_30_2024} shows that the approximation order $O(1/n_t)$ is given by $2\,\mathrm{trace}( \mathrm{E}_{p(u)}[(\partial\hat{\Psi}/\partial\theta^{\top})\partial\mathrm{E}_{g(y|u)}\{\hat{\theta}(y)-\theta_o\}/\partial\Psi_o^{\top} ] )$. Assuming that $p(u)$ follows a MVN with covariance matrix $\Sigma_u$, the Supplementary Material further derives that $O(1/n_t) = -2\mathrm{trace}\{\Sigma_u^{-1}(\partial\hat{\Psi}/\partial\theta^{\top})\mathrm{COV}(\hat{\theta})(\partial\hat{\Psi}^{\top}/\partial\theta)\}$. This result is also obtained using Eq. (7) in \cite{Zheng2021P2}, where the MVN assumption for $p(u)$ is unnecessary, and $\Sigma_u^{-1}$ is replaced by the modeling MVN precision matrix for $\Psi$, $\Sigma_{\Psi}^{-1}$. The marginal covariance of the MMLE, $\mathrm{COV}(\hat{\theta})$, is on the order of $O\{(Tn_t)^{-1}\}$. The precision matrices $\Sigma_u^{-1}$ and $\Sigma_{\Psi}^{-1}$ are typically sparse, and hence taking the trace over the RE dimension $T$ yields a term of order $O(1/n_t)$. It approaches zero only as cluster sizes increase. However, in practice, it is more common for the number of clusters to grow while cluster sizes remain bounded. Despite this, (\ref{EQ:CAIC_October_18_2024}) remains useful for model selection for two key reasons. First, the conventional AIC achieves an approximation order of $o(1)$ \citep[see, e.g.,][]{cavanaugh2019akaike}, where the degrees of freedom \citep[DF,][]{vaida2005conditional}, namely, the number of model parameters, does not increase with the number of REs or clusters. However, when the researcher's focus is on clusters instead of the population, the model DF (which corresponds to half of the terms after $-2\, \log f\{y|\hat{\theta}(y),\hat{\Psi}(y)\}$ in \ref{EQ:CAIC_October_18_2024}) increases with the number of clusters. In such cases, an approximation order of $O(1)$ is reasonable, provided the $O(1)$ term is small, as differences between mixed-effects models in cAIC are dominated by terms of order $O(T)$, such as $-2\, \log f\{y|\hat{\theta}(y),\hat{\Psi}(y)\}$, $2\,q$, and $- 2\,\mathrm{trace}\{ \ddot{l}_j^{-1}(y,\hat{\theta},\hat{\Psi})\,\ddot{l}_r(\hat{\theta},\hat{\Psi}) \}$ in (\ref{EQ:CAIC_October_18_2024}), and the competing models differ significantly in $T$. Secondly, in $O(1/n_t) = -2\mathrm{trace}\{\Sigma_u^{-1}(\partial\hat{\Psi}/\partial\theta^{\top})\mathrm{COV}(\hat{\theta})(\partial\hat{\Psi}^{\top}/\partial\theta)\}$, since $\partial\hat{\Psi}/\partial\theta^{\top}$ is typically small, its square is even smaller, making $O(1/n_t)$ negligible compared to other terms that increase with $T$.
In the next section, we use a comprehensive simulation study to demonstrate that (\ref{EQ:CAIC_October_18_2024}) provides a reasonably accurate evaluation of the cAIC.
\section{Simulation study}
\label{sec:simulation_October_21_2024}
We consider a simulation setup useful in fisheries population dynamics studies to estimate
change in population size over time ($t$) at different ages ($a$) in the population. The model includes fixed age effects, $q_a$, REs for years, $\Psi_{t}$, and age-year interaction REs, $\Psi_{t,a}$, resulting in the following linear predictor:
\begin{linenomath*}
\begin{align}\label{EQ:RE_model_October_21_2024}
\begin{split}
\eta_{t,a} &= q_a + \sigma\, \Psi_{t} + \delta\, \Psi_{t,a},
\end{split}
\end{align}
\end{linenomath*}
where $q_a$, $\sigma$ and $\delta$ are model parameters. Each year includes 6 age groups, indexed by subscript $a$, with $q_a=(-2, -1, 0, 1, 2, 3)$. The year REs, $\Psi_{t}$, follow an AR(1) process with autocorrelation parameter $\rho=0.8$ and marginal standard deviation of 1, while the age REs, $\Psi_{t,a}$, are independent standard normal variables. When generating data, $\sigma$ is fixed at 1 to serve as the reference RE strength, while $\delta$ takes values of 0.01, 0.1, 0.4, and 0.8 to explore different strengths of the age-year interaction REs. Based on $\eta_{t,a}$, we examine the performance of the proposed cAIC estimators for Gaussian, gamma, negative binomial and Tweedie observations, applying the appropriate link functions. All model parameters and REs are estimated; no parameters were fixed which is an improvement compared to some of the cAIC literature \citep[e.g.,][]{liang2008note}.
Following the approach of \cite{liang2008note}, we examine the estimates of the bias correction, BC, which is defined by $\mathrm{cAI} = \mathrm{E}_{g(y,u)}[ -2 \,\log f\{ y|\hat{\theta}(y), \hat{\Psi}(y) \} ] + \mathrm{BC}$. The procedure involves $n_{\mathrm{out}}$ (e.g., 500) outer iterations, where REs $u$ are sampled from $p(u)$, and a dataset $y$ is generated from $g(y|u)$ to compute the expectation with respect to $g(y,u)$. Within each outer iteration, $n_{\mathrm{inner}}$ (e.g., 1000) inner iterations are performed to generate $y^*$ from $g(y^*|u)$, contributing to the expectation over $y^*$ in (\ref{EQ:cAI_October_9_2024}). The true BC, averaged over the outer iterations, is denoted by $\mathrm{BC}_{\mathrm{true}}$, while the estimated BC, averaged over the outer iterations, is denoted by $\mathrm{BC}_{\mathrm{est}}$. We then assess the relative bias, $\mathrm{RB} = (\mathrm{BC}_{\mathrm{est}}-\mathrm{BC}_{\mathrm{true}})/\mathrm{BC}_{\mathrm{true}}$, for the cAIC evaluation methods.
%We examine the estimation of the correction term for estimating cAI defined in (\ref{EQ:cAI_October_9_2024}), using $-2 \log f\{y|\hat{\theta}(y),\hat{\Psi}(y)\}$, i.e., $\mathrm{cAI} + 2 \log f\{y|\hat{\theta}(y),\hat{\Psi}(y)\}$, which corresponds to twice the EDF. There are 500 outer iterations, each containing 1000 inner iterations. In each outer iteration, a set of REs is sampled from $p(u)$, followed by generating a dataset $y$ from $g(y|u)$, which is then used to obtain the estimators $\hat{\theta}(y)$ and $\hat{\Psi}(y)$, along with the estimated bias corrections using (\ref{EQ:CAIC_October_15_2024}) and (\ref{EQ:CAIC_October_18_2024}) for continuous data, and (\ref{EQ:CAIC_October_18_2024}) for non-continuous data. Here, the bias correction in each cAIC formula is the sum of all terms except for $-2 \log f\{y|\hat{\theta}(y),\hat{\Psi}(y)\}$. These 500 outer iterations evaluate the expectation over $y$ and $u$ as defined in (\ref{EQ:cAI_October_9_2024}). Within each outer iteration, using the sampled $u$, 1000 inner iterations are performed to generate $y^*$ from $g(y^*|u)$ for computing $\mathrm{E}_{g(y^*|u)} [ \log f\{ y^*|\hat{\theta}(y), \hat{\Psi}(y) \} ]$ in (\ref{EQ:cAI_October_9_2024}). The results are then averaged over the 500 outer iterations to compute cAI
\subsection{Gaussian observations}
\label{sec:Gaussian_October_21_2024}
We use the identity link function and the model for the $i$th observation $y_i$ at year $t_i$ and age $a_i$ is
\begin{linenomath*}
\begin{align}\label{EQ:yi_October_21_2024}
\begin{split}
y_i &= \eta_{t_i,a_i} + \varepsilon_i,
\end{split}
\end{align}
\end{linenomath*}
where given all the REs, the error term $\varepsilon_i$ is independently normally distributed with mean 0 and standard deviation $\sigma_e = 0.25$, 0.5 or 1. The total number of observation years is $T=50$. The sample size for each year and age point, $n_{t,a}$, is 3 or 5. When applying Eq. (\ref{EQ:CAIC_October_18_2024}), we set $n_{\mathrm{out}}=1500$ and $n_{\mathrm{inner}}=20000$. We used $n_{\mathrm{out}}=500$ and $n_{\mathrm{inner}}=1000$ when evaluating Eq. (\ref{EQ:CAIC_October_15_2024}) because of the higher computational cost of evaluating the derivatives in Eq. (\ref{EQ:derivatives_hattheta_Psi_wrt_y_October_10_2024}).
Simulation results are presented in Table \ref{tab:Gaussian_simulation_October_25_2024}. \citet{liang2008note} assumed known variances for independent Gaussian observations, allowing for an exact derivation of their bias correction formula. In their simulation study \citep[Table 1,][]{liang2008note}, the highest RBs reach values of 0.024 and 0.035. Thus, if our RBs are at or below 0.035, we can reasonably consider them small. The RBs obtained from both (\ref{EQ:CAIC_October_15_2024}) and (\ref{EQ:CAIC_October_18_2024}) for estimating cAI are indeed small, with (\ref{EQ:CAIC_October_15_2024}) generally yielding slightly smaller RBs. The RBs were similar for the different simulation choices of $n_{t,a}, \sigma_e$, and $\delta$.
\begin{table}[ht]
\centering
\caption{Relative biases (RB) in estimating the bias correction for the conditional Akaike information. The data are simulated using the linear predictor (\ref{EQ:RE_model_October_21_2024}) along with the Gaussian observational model (\ref{EQ:yi_October_21_2024}).}
\label{tab:Gaussian_simulation_October_25_2024}
\begin{tabular}{ccccc|ccccc}
\toprule
$n_{t,a}$ & $\sigma_e$ & $\delta$ & RB for (\ref{EQ:CAIC_October_18_2024}) & RB for (\ref{EQ:CAIC_October_15_2024}) & $n_{t,a}$ & $\sigma_e$ & $\delta$ & RB for (\ref{EQ:CAIC_October_18_2024}) & RB for (\ref{EQ:CAIC_October_15_2024}) \\
\midrule
3 & 1 & 0.01 & 0.003 & 0.025 & 5 & 1 & 0.01 & -0.006 & 0.019 \\
3 & 1 & 0.10 & 0.013 & 0.023 & 5 & 1 & 0.10 & 0.013 & 0.018 \\
3 & 1 & 0.40 & 0.021 & 0.007 & 5 & 1 & 0.40 & 0.025 & -0.004 \\
3 & 1 & 0.80 & 0.017 & -0.001 & 5 & 1 & 0.80 & 0.023 & -0.003 \\
3 & 0.5 & 0.01 & 0.025 & -0.002 & 5 & 0.5 & 0.01 & 0.020 & 0.001 \\
3 & 0.5 & 0.10 & 0.003 & -0.008 & 5 & 0.5 & 0.10 & 0.014 & 0.031 \\
3 & 0.5 & 0.40 & 0.023 & 0.006 & 5 & 0.5 & 0.40 & 0.022 & 0.012 \\
3 & 0.5 & 0.80 & 0.023 & 0.003 & 5 & 0.5 & 0.80 & 0.026 & -0.007 \\
3 & 0.25 & 0.01 & 0.010 & 0.002 & 5 & 0.25 & 0.01 & 0.008 & 0.002 \\
3 & 0.25 & 0.10 & 0.023 & 0.006 & 5 & 0.25 & 0.10 & 0.020 & 0.010 \\
3 & 0.25 & 0.40 & 0.022 & -0.006 & 5 & 0.25 & 0.40 & 0.026 & 0.002 \\
3 & 0.25 & 0.80 & 0.019 & -0.001 & 5 & 0.25 & 0.80 & 0.027 & 0.001 \\
\bottomrule
\end{tabular}
\end{table}
\FloatBarrier
\subsection{Gamma observations}
\label{sec:gamma_October_21_2024}
We assume that, conditional on all the REs, the observations follow independent gamma distributions.
For the gamma shape parameter, we examine a value of 3 for the skewed case and a value of 5 for the near-Gaussian case. The mean $\mu_{t,a}$ of the observations at year $t$ and age $a$ is modeled using a log-link function,
\begin{linenomath*}
\begin{align}\label{EQ:log_link_NB_October_28_2024}
\begin{split}
\log( \mu_{t,a} ) &=\eta_{t,a}.
\end{split}
\end{align}
\end{linenomath*}
The total number of years is $T=50$. When applying Eq. (\ref{EQ:CAIC_October_18_2024}), we set $n_{\mathrm{out}}=2500$ and $n_{\mathrm{inner}}=20000$. For Eq. (\ref{EQ:CAIC_October_15_2024}), due to the higher computational cost of evaluating the derivatives in Eq. (\ref{EQ:derivatives_hattheta_Psi_wrt_y_October_10_2024}), we used $n_{\mathrm{out}}=1000$ and $n_{\mathrm{inner}}=10000$.
The simulation results are shown in Table \ref{tab:Gamma_simulation_October_28_2024}. Method (\ref{EQ:CAIC_October_15_2024}) performed well with small RBs in the near-Gaussian case (shape = 5). However, it exhibited large RBs in the skewed case (shape = 3) when $\delta = 0.01$ or $0.1$, suggesting that higher-order derivatives with respect to $y$ may be necessary. In contrast, method (\ref{EQ:CAIC_October_18_2024}) performed reliably across all levels of skewness in the observational distribution, showing small RBs across all simulation scenarios. In the Supplementary Material, we provide simulation results for an alternative scenario involving gamma observations and a more complex link function, demonstrating that (\ref{EQ:CAIC_October_15_2024}) can fail for skewed distributions with intricate link functions, whereas (\ref{EQ:CAIC_October_18_2024}) exhibits significantly greater robustness.
\begin{table}[ht]
\centering
\caption{Relative biases (RB) in estimating bias correction for the conditional Akaike information, based on data simulated with the linear predictor (\ref{EQ:RE_model_October_21_2024}) and gamma observational distribution with log-link function (\ref{EQ:log_link_NB_October_28_2024}).}
\label{tab:Gamma_simulation_October_28_2024}
\begin{tabular}{ccccc|ccccc}
\toprule
$n_{t,a}$ & Shape & $\delta$ & RB for (\ref{EQ:CAIC_October_18_2024}) & RB for (\ref{EQ:CAIC_October_15_2024}) & $n_{t,a}$ & Shape & $\delta$ & RB for (\ref{EQ:CAIC_October_18_2024}) & RB for (\ref{EQ:CAIC_October_15_2024}) \\
\midrule
3 & 3 & 0.01 & 0.006 & 0.149 & 5 & 3 & 0.01 & -0.002 & 0.183 \\
3 & 3 & 0.10 & -0.016 & 0.160 & 5 & 3 & 0.10 & -0.008 & 0.137 \\
3 & 3 & 0.40 & -0.042 & 0.018 & 5 & 3 & 0.40 & -0.028 & 0.018 \\
3 & 3 & 0.80 & -0.077 & -0.059 & 5 & 3 & 0.80 & -0.034 & -0.013 \\
3 & 5 & 0.01 & 0.021 & 0.049 & 5 & 5 & 0.01 & 0.009 & 0.067 \\
3 & 5 & 0.10 & 0.007 & 0.046 & 5 & 5 & 0.10 & -0.007 & 0.060 \\
3 & 5 & 0.40 & -0.032 & -0.022 & 5 & 5 & 0.40 & -0.007 & -0.007 \\
3 & 5 & 0.80 & -0.042 & -0.050 & 5 & 5 & 0.80 & -0.017 & -0.031 \\
\bottomrule
\end{tabular}
\end{table}
\FloatBarrier
\subsection{Negative binomial observations}
\label{sec:negative_binomial_October_28_2024}
We assume that, given all the REs, the observations are distributed independently with negative binomial distributions. We apply the log-link function (\ref{EQ:log_link_NB_October_28_2024}) to model the mean $\mu_{t,a}$ of the observations at year $t$ and age $a$. The corresponding variance is defined by the dispersion relationship of the negative binomial distribution, $\mu_{t,a} + \alpha\, \mu_{t,a}^2$, where the dispersion parameter $\alpha$ is set to be 0.2 or 0.5. The total number of years is $T=50$. Because the data follow a discrete distribution, we apply only method (\ref{EQ:CAIC_October_18_2024}), with $n_{\mathrm{out}}=500$ and $n_{\mathrm{inner}}=1000$. The simulation results in Table \ref{tab:negative_binomial_simulation_October_28_2024} show that method (\ref{EQ:CAIC_October_18_2024}) achieved low RBs across all simulation scenarios.
\begin{table}[ht]
\centering
\caption{Relative biases (RB) in estimating bias correction for the conditional Akaike Information, based on data simulated with the linear predictor in Eq. (\ref{EQ:RE_model_October_21_2024}) and negative binomial observational distribution with mean defined by Eq. (\ref{EQ:log_link_NB_October_28_2024}) and dispersion parameter $\alpha$.}
\label{tab:negative_binomial_simulation_October_28_2024}
\begin{tabular}{cccc|cccc}
\toprule
$n_{t,a}$ & $\alpha$ & $\delta$ & RB for (\ref{EQ:CAIC_October_18_2024}) & $n_{t,a}$ & $\alpha$ & $\delta$ & RB for (\ref{EQ:CAIC_October_18_2024}) \\
\midrule
3 & 0.2 & 0.01 & -0.003 & 5 & 0.2 & 0.01 & 0.001 \\
3 & 0.2 & 0.10 & 0.022 & 5 & 0.2 & 0.10 & 0.028 \\
3 & 0.2 & 0.40 & -0.017 & 5 & 0.2 & 0.40 & 0.005 \\
3 & 0.2 & 0.80 & 0.011 & 5 & 0.2 & 0.80 & 0.017 \\
3 & 0.5 & 0.01 & -0.004 & 5 & 0.5 & 0.01 & -0.007 \\
3 & 0.5 & 0.10 & -0.026 & 5 & 0.5 & 0.10 & 0.048 \\
3 & 0.5 & 0.40 & -0.027 & 5 & 0.5 & 0.40 & -0.011 \\
3 & 0.5 & 0.80 & -0.045 & 5 & 0.5 & 0.80 & -0.020 \\
\bottomrule
\end{tabular}
\end{table}
\FloatBarrier
\subsection{Tweedie observations}
The Tweedie distribution \citep{jorgensen1997theory} encompasses a family of important distributions indexed by the power parameter $p$. To evaluate the performance of method (\ref{EQ:CAIC_October_18_2024}) for zero-inflated distributions, we focus on the case $1<p<2$, where the Tweedie distribution is a compound Poisson-gamma distribution that is continuous over the positive real axis and includes a point mass at zero. Specifically, we set $p=1.5$ and use dispersion parameter values $\phi=0.3$ or 0.5, applying the log-link function (\ref{EQ:log_link_NB_October_28_2024}) to model the mean. The variance of the Tweedie distribution follows the power law $\mathrm{Var}(Y_{t,a})=\phi\, \mu_{t,a}^p$. Because the observation distribution lacks differentiability across its entire support, we apply only method (\ref{EQ:CAIC_October_18_2024}). Additionally, due to the relatively slow generation of Tweedie random variables, we set $T=30$, $n_{\mathrm{out}} = 500$ and $n_{\mathrm{inner}} = 1000$. The simulation results in Table \ref{tab:Tweedie_simulation_October_28_2024} indicate that method (\ref{EQ:CAIC_October_18_2024}) achieved low RBs for Tweedie observations across all scenarios.
\begin{table}[ht]
\centering
\caption{Relative biases (RB) in estimating bias correction for the conditional Akaike Information, based on data simulated with the linear predictor in Eq. (\ref{EQ:RE_model_October_21_2024}) and Tweedie observational distribution with mean defined by Eq. (\ref{EQ:log_link_NB_October_28_2024}), power parameter $p=1.5$ and dispersion parameter $\phi$.}
\label{tab:Tweedie_simulation_October_28_2024}
\begin{tabular}{cccc|cccc}
\toprule
$n_{t,a}$ & $\phi$ & $\delta$ & RB for (\ref{EQ:CAIC_October_18_2024}) & $n_{t,a}$ & $\phi$ & $\delta$ & RB for (\ref{EQ:CAIC_October_18_2024}) \\
\midrule
3 & 0.3 & 0.01 & 0.014 & 5 & 0.3 & 0.01 & 0.031 \\
3 & 0.3 & 0.10 & -0.018 & 5 & 0.3 & 0.10 & 0.021 \\
3 & 0.3 & 0.40 & 0.005 & 5 & 0.3 & 0.40 & 0.023 \\
3 & 0.3 & 0.80 & 0.019 & 5 & 0.3 & 0.80 & 0.030 \\
3 & 0.5 & 0.01 & 0.016 & 5 & 0.5 & 0.01 & 0.014 \\
3 & 0.5 & 0.10 & 0.012 & 5 & 0.5 & 0.10 & -0.004 \\
3 & 0.5 & 0.40 & -0.004 & 5 & 0.5 & 0.40 & 0.017 \\
3 & 0.5 & 0.80 & -0.001 & 5 & 0.5 & 0.80 & 0.023 \\
\bottomrule
\end{tabular}
\end{table}
\FloatBarrier
\section{Conclusion}
\label{sec:conc}
We proposed two estimators for the conditional Akaike information criterion (cAIC) in Eqs. (\ref{EQ:CAIC_October_15_2024}) and (\ref{EQ:CAIC_October_18_2024}). For Eq. (\ref{EQ:CAIC_October_15_2024}), we derived analytical expressions for the necessary derivatives in (\ref{EQ:derivatives_hattheta_Psi_wrt_y_October_10_2024}), which can be evaluated using advanced modeling packages such as TMB. We used the relative bias (RB) in estimating the bias correction for the conditional Akaike information to evaluate the estimator's performance. The evaluation was conducted under conditions of small, upper-bounded cluster sizes (3 or 5), a common scenario in spatiotemporal modeling across various disciplines. Our simulation studies show that (\ref{EQ:CAIC_October_15_2024}) performs well for Gaussian observations with an identity link function but fails with skewed distributions (e.g., gamma distributions with small shape parameters). Additionally, (\ref{EQ:CAIC_October_15_2024}) is unsuitable for observational distributions lacking well-defined derivatives across their supports, such as discrete or zero-inflated distributions. In contrast, (\ref{EQ:CAIC_October_18_2024}) consistently performs well across all observational distributions and link functions examined in this study, highlighting its potential as a robust conditional Akaike information criterion (cAIC) for model selection across diverse applications.
Although the $O(1/n_t)$ term in (\ref{EQ:CAIC_October_18_2024}) is equal to $-2\mathrm{trace}\{\Sigma_u^{-1}(\partial\hat{\Psi}/\partial\theta^{\top})\mathrm{COV}(\hat{\theta})(\partial\hat{\Psi}^{\top}/\partial\theta)\}$ when $p(u)$ follows a multivariate normal distribution with covariance matrix $\Sigma_u$, our unpublished simulation results suggest that including this term may not improve the performance of the estimator in (\ref{EQ:CAIC_October_18_2024}). In particular, when the true value of the positive parameter $\delta$ in (\ref{EQ:RE_model_October_21_2024}) is 0, lying on the boundary of its parameter space, the variance of the maximum marginal likelihood estimator of $\delta$ cannot be estimated by the negative inverse of the Fisher information, making the $O(1/n_t)$ term difficult to evaluate. Further improvements to the method (\ref{EQ:CAIC_October_18_2024}) could focus on alternative approaches to evaluate or approximate the final term in the generic Equation (\ref{EQ:CAIC_October_18_2024}). For certain models, such as those in the exponential family, this term may also have a closed-form expression, enabling more accurate cAIC calculations for these cases.
\FloatBarrier
\section*{Funding}
Research funding to NZ was provided by the Natural Sciences and Engineering Research Council of Canada [RGPIN-2024-04746].
Research funding to NC was provided by 1) the Ocean Choice
International Industry Research Chair program at the Marine Institute
of Memorial University of Newfoundland, 2) the Ocean Frontier Institute, through an award from the Canada First Research Excellence Fund, and 3) the Natural Sciences and Engineering Research Council of Canada [RGPIN-2016-04307].
\begin{appendices}
\section{Derivation of Eqs. (\ref{EQ:derivatives_hattheta_Psi_wrt_y_October_10_2024})} \label{app:derivatives_derivation_October_30_2024}
Assume that the log-marginal likelihood $l(y,\theta)$ and log-joint likelihood $l_j(y,\theta,\Psi)$ have well-defined second-order derivatives. The maximum marginal likelihood estimator of model parameters, $\hat{\theta}$, is defined by
\begin{linenomath*}
\begin{align*}
\dfrac{\partial l(y,\hat{\theta})}{\partial\hat{\theta}} &\equiv 0,
\end{align*}
\end{linenomath*}
which is true for any data $y$. Therefore,
\begin{linenomath*}
\begin{align}\label{dtheta_dy_July_21_2024}
\begin{split}
&\dfrac{d}{dy}\dfrac{\partial l(y,\hat{\theta})}{\partial\hat{\theta}^{\top}} = 0 \Rightarrow
\dfrac{\partial^2 l(y,\hat{\theta})}{\partial y\partial\hat{\theta}^{\top}} + \dfrac{d\hat{\theta}^{\top}}{d y}\dfrac{\partial^2 l(y,\hat{\theta})}{\partial\hat{\theta}\partial\hat{\theta}^{\top}} =0 \\
&\Rightarrow \dfrac{d\hat{\theta}^{\top}}{d y} = -\dfrac{\partial^2 l(y,\hat{\theta})}{\partial y\partial\hat{\theta}^{\top}}\left\lbrace \dfrac{\partial^2 l(y,\hat{\theta})}{\partial\hat{\theta}\partial\hat{\theta}^{\top}} \right\rbrace^{-1}.
\end{split}
\end{align}
\end{linenomath*}
Similarly, the empirical Bayes RE predictor, $\hat{\Psi}(\hat{\theta})$, is defined by
\begin{linenomath*}
\begin{align*}
\dfrac{\partial l_j(y,\hat{\theta},\hat{\Psi})}{\partial\hat{\Psi}} &\equiv 0,
\end{align*}
\end{linenomath*}
which is true for any $y$ and $\hat{\theta}$. Therefore,
\begin{linenomath*}
\begin{align}\label{dPsi_dy_July_21_2024}
\begin{split}
\dfrac{d}{dy} \dfrac{\partial l_j(y,\hat{\theta},\hat{\Psi})}{\partial\hat{\Psi}^{\top}} &= 0 = \dfrac{\partial^2 l_j(y,\hat{\theta},\hat{\Psi})}{\partial y\partial\hat{\Psi}^{\top}} + \dfrac{d\hat{\theta}^{\top}}{d y}\dfrac{\partial^2 l_j(y,\hat{\theta},\hat{\Psi})}{\partial \hat{\theta}\partial\hat{\Psi}^{\top}} + \dfrac{d\hat{\Psi}^{\top}}{d y}\dfrac{\partial^2 l_j(y,\hat{\theta},\hat{\Psi})}{\partial \hat{\Psi}\partial\hat{\Psi}^{\top}} \Rightarrow\\
\dfrac{d\hat{\Psi}^{\top}}{d y} &= -\left\lbrace \dfrac{\partial^2 l_j(y,\hat{\theta},\hat{\Psi})}{\partial y\partial\hat{\Psi}^{\top}} + \dfrac{d\hat{\theta}^{\top}}{d y}\dfrac{\partial^2 l_j(y,\hat{\theta},\hat{\Psi})}{\partial \hat{\theta}\partial\hat{\Psi}^{\top}} \right\rbrace \left\lbrace \dfrac{\partial^2 l_j(y,\hat{\theta},\hat{\Psi})}{\partial \hat{\Psi}\partial\hat{\Psi}^{\top}} \right\rbrace^{-1},
\end{split}
\end{align}
\end{linenomath*}
where $d\hat{\theta}^{\top}/dy$ is given by (\ref{dtheta_dy_July_21_2024}).
\section{Derivation of Eq. (\ref{EQ:CAIC_October_15_2024})}\label{app:derivation_4_October_30_2024}
According to \cite{vaida2005conditional}, the conditional Akaike information (cAI) is defined with a new set of data $y^*$ generated independently from $g(y^*|u)$ as
\begin{linenomath*}
\begin{align}\label{cAI_defining_September_14_2024}
\mbox{cAI} = &-2\,\mathrm{E}_{g(y,u)}\mathrm{E}_{g(y^*|u)}l_c\{\hat{\theta}(y),\hat{\Psi}(y)|y^*\}\\
= &-2\,\mathrm{E}_{g(y,u)}\mathrm{E}_{g(y^*|u)}\left[ l_c(\theta_o,\Psi_o|y^*) + \dfrac{\partial l_c(\Theta_o|y^*)}{\partial\Theta_o^{\top}}\{\hat{\Theta}(y) - \Theta_o\} \right. \\
&\left. + \dfrac{1}{2}\{\hat{\Theta}(y) - \Theta_o\}^{\top}\dfrac{\partial^2 l_c(\Theta_o|y^*)}{\partial\Theta_o\partial\Theta_o^{\top}}\{\hat{\Theta}(y) - \Theta_o\} \right] \nonumber\\
= &-2\,\mathrm{E}_{g(y,u)}\left[ \vphantom{\dfrac{\partial^2 l_c(\Theta_o|y^*)}{\partial\Theta_o\partial\Theta_o^{\top}}} \mathrm{E}_{g(y^*|u)}\left\lbrace l_c(\theta_o,\Psi_o|y^*)\right\rbrace \right. \\
&\left. + \dfrac{1}{2}\{\hat{\Theta}(y) - \Theta_o\}^{\top}\mathrm{E}_{g(y^*|u)}\left\lbrace\dfrac{\partial^2 l_c(\Theta_o|y^*)}{\partial\Theta_o\partial\Theta_o^{\top}}\right\rbrace\{\hat{\Theta}(y) - \Theta_o\} \right], \nonumber
\end{align}
\end{linenomath*}
where $\hat{\Theta}(y)$ is a function of $y$ which is independent of $y^*$. The observed $l_c$ is
\begin{linenomath*}
\begin{align}\label{lc_August_4_2024}
\begin{split}
l_c(\hat{\theta},\hat{\Psi}|y) &= l_c(\Theta_o|y) + (\hat{\Theta} - \Theta_o)^{\top}\dfrac{\partial l_c(\Theta_o|y)}{\partial\Theta_o}+ \dfrac{1}{2}(\hat{\Theta} - \Theta_o)^{\top}\dfrac{\partial^2 l_c(\Theta_o|y)}{\partial\Theta_o\partial\Theta_o^{\top}}(\hat{\Theta} - \Theta_o).
\end{split}
\end{align}
\end{linenomath*}
Let $\mathrm{E}_{g(y|u)}(y)=\mu_y$, and $\mathrm{E}_{g(y,u)}(\hat{\Theta}-\Theta_o)\approx0$ according to \cite{zheng2021frequentist}. By denoting the marginal expectation $\mathrm{E}_{g(y,u)}(\cdot)$ as $\mathrm{E}(\cdot)$, We evaluate
\begin{linenomath*}
\begin{align}\label{expectation_lc_September_14_2024}
\begin{split}
-2\mathrm{E}\{l_c(\hat{\theta},\hat{\Psi}|y)\} &= -2\mathrm{E}\left\lbrace l_c(\Theta_o|y) + (\hat{\Theta} - \Theta_o)^{\top}\dfrac{\partial l_c(\Theta_o|y)}{\partial\Theta_o}\right. \\
&\left.\hspace{4.5mm} + \dfrac{1}{2}(\hat{\Theta} - \Theta_o)^{\top}\dfrac{\partial^2 l_c(\Theta_o|y)}{\partial\Theta_o\partial\Theta_o^{\top}}(\hat{\Theta} - \Theta_o)\right\rbrace \\
&= -2\mathrm{E}\left\lbrace l_c(\Theta_o|y) + (y-\mu_y)^{\top}\dfrac{\partial\hat{\Theta}^{\top}(\mu_y)}{\partial \mu_y}\left. \dfrac{\partial^2 l_c(\Theta_o|y)}{\partial\Theta_o\partial y^{\top}}\right|_{y=\mu_y}(y-\mu_y) \right. \\
&\left.\hspace{4.5mm} + \dfrac{1}{2}(\hat{\Theta} - \Theta_o)^{\top}\dfrac{\partial^2 l_c(\Theta_o|y)}{\partial\Theta_o\partial\Theta_o^{\top}}(\hat{\Theta} - \Theta_o)\right\rbrace.
\end{split}
\end{align}
\end{linenomath*}
If we neglect the difference between the second order terms of (\ref{cAI_defining_September_14_2024}) and (\ref{expectation_lc_September_14_2024}) (our unpublished numerical study also indicates that the difference is small), the bias from using $-2l_c(\hat{\theta},\hat{\Psi}|y)$ to estimate cAI formulated by (\ref{cAI_defining_September_14_2024}) is
\begin{linenomath*}
\begin{align*}%\label{lc_August_4_2024}
\begin{split}
-2\, \mathrm{trace}\left\lbrace \dfrac{\partial\hat{\Theta}^{\top}(y)}{\partial y}\left. \dfrac{\partial^2 l_c(\Theta_o|y)}{\partial\Theta_o\partial y^{\top}}\right|_{y=\mu_y}\mathrm{Cov}(y) \right\rbrace .
\end{split}
\end{align*}
\end{linenomath*}
\section{Derivation of Eq. (\ref{EQ:CAIC_October_18_2024})}\label{app:derivation_6_October_30_2024}
We continue with Eqs. (\ref{cAI_defining_September_14_2024}) and (\ref{lc_August_4_2024}), which hold for any set of observations.
For clarity, we denote the fixed effects (parameters) in $l_c$ as $\phi$. Then the log-marginal distribution of the REs $\Psi$ is written as $l_r(\tau,\Psi)$, where $\tau$ represents the associated fixed parameters. By assumption, $f(y|\phi_o,\Psi_o)=g(y|u)$. We examine the $i$th component of $\phi$ and note that the estimator $\hat{\Theta}$ depends solely on $y$, with no dependence on $\Theta_o$,
\begin{linenomath*}
\begin{align}
\begin{split}
&\mathrm{E}_{p(u)}\mathrm{E}_{g(y|u)}\left\lbrace (\hat{\phi}_i-\phi_{o,i})\dfrac{\partial l_c(\phi_o,\Psi_o|y)}{\partial\phi_{o,i}}\right\rbrace \\
&= \mathrm{E}_{p(u)}\int (\hat{\phi}_i-\phi_{o,i})\dfrac{\partial f(y|\phi_o,\Psi_o)/\partial\phi_{o,i}}{f(y|\phi_o,\Psi_o)}g(y|u) dy\\
&= \mathrm{E}_{p(u)}\int (\hat{\phi}_i-\phi_{o,i})\dfrac{\partial f(y|\phi_o,\Psi_o)}{\partial\phi_{o,i}} dy\\
&= \mathrm{E}_{p(u)}\left\lbrace \dfrac{\partial }{\partial\phi_{o,i}}\int (\hat{\phi}_i-\phi_{o,i})f(y|\phi_o,\Psi_o) dy - \int \dfrac{\partial(\hat{\phi}_i-\phi_{o,i})}{\partial\phi_{o,i}}f(y|\phi_o,\Psi_o) dy \right\rbrace \\
&= \dfrac{\partial }{\partial\phi_{o,i}}\mathrm{E}_{p(u)}\mathrm{E}_{g(y|u)}(\hat{\phi}_i-\phi_{o,i}) + 1 = 1.
\end{split}
\end{align}
\end{linenomath*}
Here we can exchange $\mathrm{E}_{p(u)}$ and $\partial/\partial\phi_{o,i}$ because $p(u)$ does not depend on $\phi_o$. We also applied $\mathrm{E}(\hat{\phi}_i-\phi_{o,i})=0$. Therefore,
\begin{linenomath*}
\begin{align}
\begin{split}
\mathrm{E}_{p(u)}\mathrm{E}_{g(y|u)}\left\lbrace (\hat{\phi}-\phi_o)^{\top}\dfrac{\partial l_c(\phi_o,\Psi_o|y)}{\partial\phi_o}\right\rbrace &= p_c,
\end{split}
\end{align}
\end{linenomath*}
where $p_c$ is the number of parameters in $l_c(\phi_o,\Psi_o|y)$, or equivalently the length of $\phi_o$.
Assume that $\Psi_o$ is modeled with a multivariate normal distribution with mean $\mu_{\Psi}$ and covariance $\Sigma$. Then, according to the the conditional inference outlined in \cite{Zheng2021P2},
\begin{linenomath*}
\begin{align}\label{EQ:Psi_1st_order_term_November_2_2024}
\begin{split}
&\mathrm{E}_{p(u)}\mathrm{E}_{g(y|u)}\left\lbrace (\hat{\Psi}_i-\Psi_{o,i})\dfrac{\partial l_c(\phi_o,\Psi_o|y)}{\partial\Psi_{o,i}}\right\rbrace \\
&= \mathrm{E}_{p(u)}\int (\hat{\Psi}_i-\Psi_{o,i})\dfrac{\partial f(y|\phi_o,\Psi_o)/\partial\Psi_{o,i}}{f(y|\phi_o,\Psi_o)}g(y|u) dy\\
&= \mathrm{E}_{p(u)}\int (\hat{\Psi}_i-\Psi_{o,i})\dfrac{\partial f(y|\phi_o,\Psi_o)}{\partial\Psi_{o,i}} dy\\
&= \mathrm{E}_{p(u)}\left\lbrace \dfrac{\partial }{\partial\Psi_{o,i}}\int (\hat{\Psi}_i-\Psi_{o,i})f(y|\phi_o,\Psi_o) dy - \int \dfrac{\partial(\hat{\Psi}_i-\Psi_{o,i})}{\partial\Psi_{o,i}}f(y|\phi_o,\Psi_o) dy \right\rbrace \\
&= \mathrm{E}_{p(u)}\dfrac{\partial }{\partial\Psi_{o,i}}\mathrm{E}_{g(y|u)}(\hat{\Psi}_i-\Psi_{o,i}) + 1\\
&=-\mathrm{E}_{p(u)}\dfrac{\partial }{\partial\Psi_{o,i}}\mathbf{0}_i^{\top}\ddot{l}^{-1}_j(\phi_o,\tau_o,\Psi_o|y)\ddot{l}_r(\tau_o,\Psi_o)(\Psi_o-\mu_{\Psi}) + 1 \\
&\hspace{4.5mm}+ \mathrm{E}_{p(u)}\dfrac{\partial }{\partial\Psi_{o,i}}\mathrm{E}_{g(y|u)}(\hat{\theta}-\theta_o)^{\top}\dfrac{\partial\hat{\Psi}_i}{\partial\theta_o}\\
&=-\mathrm{E}_{p(u)}\mathbf{0}_i^{\top}\ddot{l}^{-1}_j(\phi_o,\tau_o,\Psi_o|y)\ddot{l}_r(\tau_o,\Psi_o)\mathbf{0}_i + 1 + \mathbf{0}_i^{\top} \mathrm{E}_{p(u)}\dfrac{\partial }{\partial\Psi_o}\mathrm{E}_{g(y|u)}(\hat{\theta}-\theta_o)^{\top}\dfrac{\partial\hat{\Psi}^{\top}}{\partial\theta_o} \mathbf{0}_i,
\end{split}
\end{align}
\end{linenomath*}
where the double dots denotes hessian with respect to $\Psi_o$, and $\mathbf{0}_i$ is a vector of 0's with the $i$th element being 1. Therefore,
\begin{linenomath*}
\begin{align}
\begin{split}
\mathrm{E}_{p(u)}\mathrm{E}_{g(y|u)}\left\lbrace (\hat{\Psi}-\Psi_o)^{\top}\dfrac{\partial l_c(\phi_o,\Psi_o|y)}{\partial\Psi_o}\right\rbrace = &- \mbox{trace}\left\lbrace\ddot{l}^{-1}_j(\phi_o,\tau_o,\Psi_o|y) \ddot{l}_r(\tau_o,\Psi_o)\right\rbrace + q \\
&\hspace{-25.5mm}+ \mbox{trace}\left\lbrace \mathrm{E}_{p(u)}\dfrac{\partial }{\partial\Psi_o}\mathrm{E}_{g(y|u)}(\hat{\theta}-\theta_o)^{\top}\dfrac{\partial\hat{\Psi}^{\top}}{\partial\theta_o} \right\rbrace\\
&\hspace{-30mm}= - \mbox{trace}\left\lbrace\ddot{l}^{-1}_j(\phi_o,\tau_o,\Psi_o|y) \ddot{l}_r(\tau_o,\Psi_o)\right\rbrace + q + O(1/n_t),
\end{split}
\end{align}
\end{linenomath*}
where $q$ is the number of random effects, and we prove the $O(1/n_t)$ order in the Supplementary Material.
Thus,
\begin{linenomath*}
\begin{align}\label{Elc_August_4_2024}
\begin{split}
\mathrm{E}\left\lbrace l_c(\hat{\phi},\hat{\Psi}|y)\right\rbrace &= \mathrm{E}\left\lbrace l_c(\Theta_o|y)\right\rbrace + \mathrm{E}\left\lbrace (\hat{\Theta} - \Theta_o)^{\top}\dfrac{\partial l_c(\Theta_o|y)}{\partial\Theta_o}\right\rbrace\\ &\hspace{4.5mm}+\mathrm{E}\left\lbrace \dfrac{1}{2}(\hat{\Theta} - \Theta_o)^{\top}\dfrac{\partial^2 l_c(\Theta_o|y)}{\partial\Theta_o\partial\Theta_o^{\top}}(\hat{\Theta} - \Theta_o)\right\rbrace \\
&= \mathrm{E}\left\lbrace l_c(\Theta_o|y)\right\rbrace + p_c + q - \mbox{trace}\left\lbrace \ddot{l}^{-1}_j(\phi_o,\tau_o,\Psi_o|y)\ddot{l}_r(\tau_o,\Psi_o)\right\rbrace \\
&\hspace{4.5mm}+ \mathrm{E}\left\lbrace \dfrac{1}{2}(\hat{\Theta} - \Theta_o)^{\top}\dfrac{\partial^2 l_c(\Theta_o|y)}{\partial\Theta_o\partial\Theta_o^{\top}}(\hat{\Theta} - \Theta_o)\right\rbrace + O(1/n_t).
\end{split}
\end{align}
\end{linenomath*}
The bias for using $-2\,l_c(\hat{\phi},\hat{\Psi}|y)$ to estimate cAI is then
\begin{linenomath*}
\begin{align}\label{Elc_August_4_2024}
\begin{split}
&-2\,\mathrm{E}\left\lbrace l_c(\hat{\phi},\hat{\Psi}|y)\right\rbrace - \mathrm{cAI} = -2\,\mathrm{E}\left\lbrace l_c(\Theta_o|y)\right\rbrace -2\, p_c -2\,q + O(1/n_t) \\
&+2\, \mbox{trace}\left\lbrace \ddot{l}^{-1}_j(\phi_o,\tau_o,\Psi_o|y)\ddot{l}_r(\tau_o,\Psi_o)\right\rbrace -\mathrm{E}\left\lbrace (\hat{\Theta} - \Theta_o)^{\top}\dfrac{\partial^2 l_c(\Theta_o|y)}{\partial\Theta_o\partial\Theta_o^{\top}}(\hat{\Theta} - \Theta_o)\right\rbrace\\
& +2\,\mathrm{E}_{g(y,u)}\left[ \mathrm{E}_{g(y^*|u)}\left\lbrace l_c(\phi_o,\Psi_o|y^*)\right\rbrace + \dfrac{1}{2}(\hat{\Theta} - \Theta_o)^{\top}\mathrm{E}_{g(y^*|u)}\left\lbrace\dfrac{\partial^2 l_c(\Theta_o|y^*)}{\partial\Theta_o\partial\Theta_o^{\top}}\right\rbrace(\hat{\Theta} - \Theta_o) \right]\\
&= -2\, p_c -2\,q +2\, \mbox{trace}\left\lbrace\ddot{l}^{-1}_j(\phi_o,\tau_o,\Psi_o|y) \ddot{l}_r(\tau_o,\Psi_o)\right\rbrace + O(1/n_t).
\end{split}
\end{align}
\end{linenomath*}
Here we neglect the difference between the second order terms.
\end{appendices}
\bibliographystyle{Chicago}
\bibliography{mybib_marine}
\bigskip
\newpage
\begin{center}
{\large\bf SUPPLEMENTARY MATERIAL}
\end{center}
\begin{description}
\item[Derivation of the $O(1/n_t)$ approximation order:] %Brief description. (file type)
Assume that $p(u)$ is multivariate normal with mean $\mu_u$ and covariance matrix $\Sigma_u$. According to Stein's lemma \citep{ingersoll1987theory},
\begin{linenomath*}
\begin{align}\label{EQ:Ont_term_November_2_2024}
\begin{split}
\mathrm{E}_{p(u)}\dfrac{\partial }{\partial\Psi_o}\mathrm{E}_{g(y|u)}(\hat{\theta}-\theta_o)^{\top}\dfrac{\partial\hat{\Psi}_i}{\partial\theta_o} &= \Sigma_u^{-1}\mathrm{E}_{g(y,u)}\left\lbrace (\Psi_o-\mu_u)(\hat{\theta}-\theta_o)^{\top}\right\rbrace \dfrac{\partial\hat{\Psi}_i}{\partial\theta_o}\\
&= -\Sigma_u^{-1} \dfrac{\partial\hat{\Psi}}{\partial\theta_o^{\top}} \mathrm{COV}(\hat{\theta}) \dfrac{\partial\hat{\Psi}_i}{\partial\theta_o},
\end{split}
\end{align}
\end{linenomath*}
according to Eq. (30) in \cite{zheng2021frequentist}. Thus, by Eq. (\ref{EQ:Psi_1st_order_term_November_2_2024}), the $O(1/n_t)$ is equal to
\begin{linenomath*}
\begin{align}\label{EQ:Ont_term_November_2_2024}
\begin{split}
\mathrm{trace}\left\lbrace \mathrm{E}_{p(u)}\dfrac{\partial }{\partial\Psi_o}\mathrm{E}_{g(y|u)}(\hat{\theta}-\theta_o)^{\top}\dfrac{\partial\hat{\Psi}^{\top}}{\partial\theta_o} \right\rbrace &= -\mathrm{trace}\left\lbrace \Sigma_u^{-1} \dfrac{\partial\hat{\Psi}}{\partial\theta_o^{\top}} \mathrm{COV}(\hat{\theta}) \dfrac{\partial\hat{\Psi}^{\top}}{\partial\theta_o} \right\rbrace.
\end{split}
\end{align}
\end{linenomath*}
\item[A gamma observation example with more complicated link function:]
We assume that, conditional on all the REs, the observations follow independent gamma distributions.
For the gamma shape parameter, we examine a value of 3 for the skewed case and a value of 5 for the near-Gaussian case. We model the gamma scale parameter $t$ and age $a$, $\mbox{Scale}_{t,a}$, using a shifted and scaled logit transformation,
\begin{linenomath*}
\begin{align*}%\label{EQ:gamma_shape_October_27_2024}
\begin{split}
\mbox{Scale}_{t,a} &= 0.5 + \dfrac{1.5\exp(\eta_{t,a})}{1+\exp(\eta_{t,a})}.
\end{split}
\end{align*}
\end{linenomath*}
This parameter structure constrains the standard deviation of the observations to fall between 0.87 and 6.32. The total number of years is $T=50$. When applying Eq. (\ref{EQ:CAIC_October_18_2024}), we set $n_{\mathrm{out}}=1500$ and $n_{\mathrm{inner}}=15000$. For Eq. (\ref{EQ:CAIC_October_15_2024}), due to the higher computational cost of evaluating the derivatives in Eq. (\ref{EQ:derivatives_hattheta_Psi_wrt_y_October_10_2024}), we used $n_{\mathrm{out}}=1000$ and $n_{\mathrm{inner}}=10000$.
The simulation results are shown in Table \ref{tab:Gamma_simulation_logit_October_28_2024}. Method (\ref{EQ:CAIC_October_15_2024}) performed well, with small RBs in the near-Gaussian case (shape = 5), but failed with large RBs in the skewed case (shape = 3), where higher-order derivatives with respect to $y$ may be required. In contrast, method (\ref{EQ:CAIC_October_18_2024}) performed reliably across all levels of skewness in the observational distribution, showing small RBs across all simulation scenarios.
\begin{table}[ht]
\centering
\caption{Relative biases (RB) in estimating bias correction for the conditional Akaike information, based on data simulated with the linear predictor (\ref{EQ:RE_model_October_21_2024}) and gamma observational distribution with shape parameter modeled via the shifted and scaled logit transformation.}
\label{tab:Gamma_simulation_logit_October_28_2024}
\begin{tabular}{ccccc|ccccc}
\toprule
$n_{t,a}$ & Shape & $\delta$ & RB for (\ref{EQ:CAIC_October_18_2024}) & RB for (\ref{EQ:CAIC_October_15_2024}) & $n_{t,a}$ & Shape & $\delta$ & RB for (\ref{EQ:CAIC_October_18_2024}) & RB for (\ref{EQ:CAIC_October_15_2024}) \\
\midrule
3 & 3 & 0.01 & 0.019 & 0.260 & 5 & 3 & 0.01 & 0.055 & 0.322 \\
3 & 3 & 0.10 & 0.010 & 0.290 & 5 & 3 & 0.10 & 0.065 & 0.197 \\
3 & 3 & 0.40 & 0.023 & 0.245 & 5 & 3 & 0.40 & 0.016 & 0.161 \\
3 & 3 & 0.80 & 0.018 & 0.167 & 5 & 3 & 0.80 & 0.037 & 0.171 \\
3 & 5 & 0.01 & 0.028 & 0.047 & 5 & 5 & 0.01 & 0.042 & 0.029 \\
3 & 5 & 0.10 & 0.043 & 0.087 & 5 & 5 & 0.10 & 0.032 & 0.045 \\
3 & 5 & 0.40 & 0.049 & 0.058 & 5 & 5 & 0.40 & 0.016 & 0.056 \\
3 & 5 & 0.80 & 0.046 & 0.064 & 5 & 5 & 0.80 & 0.058 & 0.025 \\
\bottomrule
\end{tabular}
\end{table}
%\FloatBarrier
\clearpage
\item[R and TMB Code for the Gaussian simulation example in Sec. \ref{sec:Gaussian_October_21_2024}:] The codes include simulation\_Gaussian\_example.R, fit.cpp, fit\_dataparameters.cpp, and fit\_sim.cpp. simulation\_Gaussian\_example.R includes simulation setup, and the function cAIC\_calc() that implements Eqs. (\ref{EQ:CAIC_October_15_2024}) and (\ref{EQ:CAIC_October_18_2024}) to evaluate the bias correction (BC) term in cAIC. The outputs of this function are cAIC using (\ref{EQ:CAIC_October_18_2024}), BC using (\ref{EQ:CAIC_October_15_2024}) and BC using (\ref{EQ:CAIC_October_18_2024}). fit.cpp serves as the main code for model fitting and data generation. fit\_dataparameters.cpp is a revision to fit.cpp by declaring data as parameters for implementing (\ref{EQ:CAIC_October_15_2024}). fit\_sim.cpp evaluate the integration over $y^*$ in Eq. (\ref{EQ:cAI_October_9_2024}) by generating and taking average over $y^*$ for evaluating the true BC.
\begin{figure}[!h]
\centering
\includegraphics[width=1\linewidth]{R_TMB_codes_for_Gaussian_example-1}
\end{figure}
\begin{figure}[!h]
\centering
\includegraphics[width=1\linewidth]{R_TMB_codes_for_Gaussian_example-2}
\end{figure}
\begin{figure}[!h]
\centering
\includegraphics[width=1\linewidth]{R_TMB_codes_for_Gaussian_example-3}
\end{figure}
\begin{figure}[!h]
\centering
\includegraphics[width=1\linewidth]{R_TMB_codes_for_Gaussian_example-4}
\end{figure}
\begin{figure}[!h]
\centering
\includegraphics[width=1\linewidth]{R_TMB_codes_for_Gaussian_example-5}
\end{figure}
\begin{figure}[!h]
\centering
\includegraphics[width=1\linewidth]{R_TMB_codes_for_Gaussian_example-6}
\end{figure}
\begin{figure}[!h]
\centering
\includegraphics[width=1\linewidth]{R_TMB_codes_for_Gaussian_example-7}
\end{figure}
\begin{figure}[!h]
\centering
\includegraphics[width=1\linewidth]{R_TMB_codes_for_Gaussian_example-8}
\end{figure}
\begin{figure}[!h]
\centering
\includegraphics[width=1\linewidth]{R_TMB_codes_for_Gaussian_example-9}
\end{figure}
\begin{figure}[!h]
\centering
\includegraphics[width=1\linewidth]{R_TMB_codes_for_Gaussian_example-10}
\end{figure}
\begin{figure}[!h]
\centering
\includegraphics[width=1\linewidth]{R_TMB_codes_for_Gaussian_example-11}
\end{figure}
\begin{figure}[!h]
\centering
\includegraphics[width=1\linewidth]{R_TMB_codes_for_Gaussian_example-12}
\end{figure}
\begin{figure}[!h]
\centering
\includegraphics[width=1\linewidth]{R_TMB_codes_for_Gaussian_example-13}
\end{figure}
%\includepdf[pages={1-last}]{R_TMB_codes_for_Gaussian_example.pdf}
%\item[R-package for MYNEW routine:] R-package ÒMYNEWÓ containing code to perform the diagnostic methods described in the article. The package also contains all datasets used as examples in the article. (GNU zipped tar file)
%\item[HIV data set:] Data set used in the illustration of MYNEW method in Section~ 3.2. (.txt file)
\end{description}
%\section{BibTeX}
%We hope you've chosen to use BibTeX!\ If you have, please feel free to use the package natbib with any bibliography style you're comfortable with. The .bst file agsm has been included here for your convenience.
\end{document}
\begin{thebibliography}{}
\bibitem[\protect\citeauthoryear{Akaike}{Akaike}{1973}]{Akaike1973AIC}
Akaike, H. (1973).
\newblock Information theory and an extension of the maximum likelihood
principle.
\newblock In B.~Petrov and F.~Csaki (Eds.), {\em Second International Symposium
on Information Theory}, pp.\ 267â81. Budapest: Akademiai Kiado.
\bibitem[\protect\citeauthoryear{Cadigan, Robertson, Nirmalkanna, and
Zheng}{Cadigan et~al.}{2022}]{cadigan2022complex}
Cadigan, N., M.~D. Robertson, K.~Nirmalkanna, and N.~Zheng (2022).
\newblock The complex relationship between weight and length of atlantic cod
off the south coast of newfoundland.
\newblock {\em Canadian Journal of Fisheries and Aquatic Sciences\/}~{\em
79\/}(11), 1798--1819.
\bibitem[\protect\citeauthoryear{Cavanaugh and Neath}{Cavanaugh and
Neath}{2019}]{cavanaugh2019akaike}
Cavanaugh, J.~E. and A.~A. Neath (2019).
\newblock The akaike information criterion: Background, derivation, properties,
application, interpretation, and refinements.
\newblock {\em Wiley Interdisciplinary Reviews: Computational
Statistics\/}~{\em 11\/}(3), e1460.
\bibitem[\protect\citeauthoryear{Hooten and Hobbs}{Hooten and
Hobbs}{2015}]{hooten2015guide}
Hooten, M.~B. and N.~T. Hobbs (2015).
\newblock A guide to bayesian model selection for ecologists.
\newblock {\em Ecological monographs\/}~{\em 85\/}(1), 3--28.
\bibitem[\protect\citeauthoryear{Ingersoll}{Ingersoll}{1987}]{ingersoll1987theory}
Ingersoll, J.~E. (1987).
\newblock {\em Theory of financial decision making}, Volume~3.
\newblock Rowman \& Littlefield.
\bibitem[\protect\citeauthoryear{Jorgensen}{Jorgensen}{1997}]{jorgensen1997theory}
Jorgensen, B. (1997).
\newblock {\em The theory of dispersion models}.
\newblock Chapman and Hall, London.
\bibitem[\protect\citeauthoryear{Kass and Steffey}{Kass and
Steffey}{1989}]{kass1989approximate}
Kass, R.~E. and D.~Steffey (1989).
\newblock Approximate bayesian inference in conditionally independent
hierarchical models (parametric empirical bayes models).
\newblock {\em Journal of the American Statistical Association\/}~{\em
84\/}(407), 717--726.
\bibitem[\protect\citeauthoryear{Kristensen, Nielsen, Berg, Skaug, and
Bell}{Kristensen et~al.}{2016}]{kristensen2016tmb}
Kristensen, K., A.~Nielsen, C.~W. Berg, H.~Skaug, and B.~Bell (2016).
\newblock Tmb: automatic differentiation and laplace approximation.
\newblock {\em Journal of Statistical Software\/}~{\em 70\/}(5), 1â21.
\bibitem[\protect\citeauthoryear{Liang, Wu, and Zou}{Liang
et~al.}{2008}]{liang2008note}
Liang, H., H.~Wu, and G.~Zou (2008).
\newblock A note on conditional aic for linear mixed-effects models.
\newblock {\em Biometrika\/}~{\em 95\/}(3), 773--778.
\bibitem[\protect\citeauthoryear{Lindstrom and Bates}{Lindstrom and
Bates}{1990}]{lindstrom1990nonlinear}
Lindstrom, M.~J. and D.~M. Bates (1990).
\newblock Nonlinear mixed effects models for repeated measures data.
\newblock {\em Biometrics\/}, 673--687.
\bibitem[\protect\citeauthoryear{{R Core Team}}{{R Core
Team}}{2022}]{r2018citation}
{R Core Team} (2022).
\newblock {\em R: A Language and Environment for Statistical Computing}.
\newblock Vienna, Austria: R Foundation for Statistical Computing.
\bibitem[\protect\citeauthoryear{Vaida and Blanchard}{Vaida and
Blanchard}{2005}]{vaida2005conditional}
Vaida, F. and S.~Blanchard (2005).
\newblock Conditional akaike information for mixed effects models.
\newblock {\em Biometrika\/}~{\em 92\/}(2), 351--370.
\bibitem[\protect\citeauthoryear{Wood, Pya, and S{\"a}fken}{Wood
et~al.}{2016}]{wood2016smoothing}
Wood, S.~N., N.~Pya, and B.~S{\"a}fken (2016).
\newblock Smoothing parameter and model selection for general smooth models.
\newblock {\em Journal of the American Statistical Association\/}~{\em
111\/}(516), 1548--1563.
\bibitem[\protect\citeauthoryear{Yu and Yau}{Yu and
Yau}{2012}]{yu2012conditional}
Yu, D. and K.~K. Yau (2012).
\newblock Conditional akaike information criterion for generalized linear mixed
models.
\newblock {\em Computational Statistics \& Data Analysis\/}~{\em 56\/}(3),
629--644.
\bibitem[\protect\citeauthoryear{Zheng and Cadigan}{Zheng and
Cadigan}{2021}]{zheng2021frequentist}
Zheng, N. and N.~Cadigan (2021).
\newblock Frequentist delta-variance approximations with mixed-effects models
and tmb.
\newblock {\em Computational Statistics \& Data Analysis\/}~{\em 160}, 107227.
\bibitem[\protect\citeauthoryear{Zheng and Cadigan}{Zheng and
Cadigan}{2023}]{Zheng2021P2}
Zheng, N. and N.~Cadigan (2023).
\newblock Frequentist conditional variance for nonlinear mixed-effects models.
\newblock {\em Journal of Statistical Theory and Practice\/}~{\em 17\/}(1),
1--30.
\end{thebibliography}