Randomization inference for treatment effects on survival outcomes

The Kaplan–Meier plot and the log-rank test are common tools used to analyze time-to-event data in randomized clinical trials. The Kaplan–Meier plot conveys the full shape of each arm’s survival curve, while the log-rank test provides a $p$-value for the null hypothesis of no treatment effect. Neither, however, directly quantify the overall magnitude of the treatment effect, an important quantity when trying to establish both statistical and clinical significance. Practitioners typically fill this gap by reporting a hazard ratio from a Cox proportional-hazards model [1]. The Cox hazard ratio is well understood, but it relies on the proportional-hazards assumption, which may be violated in practice and which is not required for the validity of either the Kaplan–Meier estimator or the log-rank test. Accelerated failure time (AFT) models [11] offer an alternative effect-size summary, but likewise require assumptions about the data-generating process.

A potential alternative is to report effect-size summaries that, like the log-rank test itself, require no parametric model and are valid under the randomization distribution alone. 4 established a finite-population randomization-inference foundation for the log-rank test under noninformative censoring, showing that the log-rank statistic is asymptotically standard normal conditional on the potential outcomes under the sharp null of no treatment effect. Their framework accommodates Bernoulli randomized designs and imposes only mild regularity conditions on the joint distribution of event and censoring times. Building on 4, we observe that their result extends to a family of sharp null hypotheses of constant treatment effects, both additive and multiplicative. Typically, randomization-based inference for a sharp null hypothesis involves transforming the potential outcomes [7]; we extend this transformation to the potential censoring times as well, which ensures that the realized time at risk and event indicator are fully determined under the null. Inverting the resulting tests yields confidence intervals (CIs) for two complementary scalar summaries of the treatment effect. The first is an additive shift parameter, $c$. Under the null, $H_{c}$, treatment adds $c$ time units to every unit’s event time. The CI for $c$ is expressed on the original time scale and answers the question “by how much did treatment delay the event?” The second is a multiplicative factor, $\rho$. Under the null, $H_{\rho}$, treatment multiplies every unit’s event time by $\rho$. This parameter targets the same estimand as the acceleration factor in AFT models, but unlike model-based AFT inference, its validity derives from the randomization distribution alone, requiring no assumption on the event time distribution or the censoring beyond that censoring times are independent and identically distributed. Both CIs are computed by evaluating the log-rank $p$-value over a fine grid of candidate parameter values and retaining those not rejected at the $\alpha$-level.

The remainder of the paper is organized as follows. Section 2 introduces the potential outcomes setup and assumptions from 4. Section 3 derives the tests for sharp nulls of additive and multiplicative constant effects. Section 4 presents the CI construction and discusses computation and interpretation. Section 5 includes simulation results demonstrating the operating characteristics in comparison to parametric model-based methods. Section 6 applies the method to real trial data and provides R code for ease of implementation. Finally, Section 7 concludes with discussion of extensions and limitations.

We first introduce the setup, notation and assumptions from 4. We consider an experiment on $n$ units with two treatment arms. For each unit $i$, let $T_{i}(1)$ and $T_{i}(0)$ denote the potential event times under treatment and control, and $C_{i}(1)$ and $C_{i}(0)$ denote the potential censoring times under treatment and control, respectively. Let $\boldsymbol{T}(1)=\bigl(T_{1}(1),\ldots,T_{n}(1)\bigr)^{\top}$, $\boldsymbol{T}(0)=\bigl(T_{1}(0),\ldots,T_{n}(0)\bigr)^{\top}$, $\boldsymbol{C}(1)=\bigl(C_{1}(1),\ldots,C_{n}(1)\bigr)^{\top}$, and $\boldsymbol{C}(0)=\bigl(C_{1}(0),\ldots,C_{n}(0)\bigr)^{\top}$.

Let $Z_{i}\in\{0,1\}$ be the treatment assignment for unit $i$, where $Z_{i}=1$ if unit $i$ receives the active treatment and $Z_{i}=0$ otherwise. The observed time at risk and the observed event indicator is

In other words, conditional on all potential event and censoring times, treatment assignments are i.i.d. across units, each with probability $p_{1}$. This formalizes randomization, meaning treatment assignment is independent of both potential outcomes and potential censoring times.

Second, we consider the distribution of $(\boldsymbol{C}(1),\boldsymbol{C}(0))$, also called the censoring mechanism. We focus on the case of noninformative i.i.d. censoring.

This holds, for example, when censoring is due to administrative end of study or when subjects withdraw from the study for reasons unrelated to their outcome status. It would be violated if, for instance, subjects in worse health were more likely to drop out.

An $\alpha$-level test of $H_{c}$ (and $H_{\rho}$, respectively) is obtained by rejecting when $|\textsc{LR}_{c}|>z_{\alpha/2}$ (or $|\textsc{LR}_{\rho}|>z_{\alpha/2}$), where $z_{\alpha/2}$ is the upper $\alpha/2$ quantile of a standard normal distribution. By test inversion, a $(1-\alpha)$ confidence set for the true additive shift $c^{*}$ is

where for $c<0$, $\textsc{LR}_{c}$ is computed after switching treatment and control labels, as discussed in Remark 1. A $(1-\alpha)$ confidence set for the true multiplicative factor $\rho^{*}$ is

We conducted a simulation study to evaluate the finite-sample operating characteristics of the proposed randomization-based confidence intervals for the multiplicative effect parameter $\rho$, and to compare their performance with standard parametric AFT model-based inference.

We considered two data-generating processes to compare the approaches when the parametric model was correctly specified and misspecified. In the first data-generating mechanism, baseline event times were drawn from a Weibull distribution,

corresponding to a setting in which the Weibull AFT model is correctly specified. In the second mechanism, baseline event times were drawn from a log-logistic distribution,

which induces misspecification of the Weibull AFT likelihood.

In both cases, treatment assignment was generated as $Z\sim\mathrm{Bernoulli}(0.5)$, and a constant multiplicative treatment effect was imposed via $T(1)=T(0)\rho$. We considered two values of the treatment effect, $\rho\in\{1.0,1.25\}$, where $\rho=1$ corresponds to the null hypothesis of no treatment effect and $\rho=1.25$ represents a beneficial effect that increases event times under treatment. Independent right censoring was generated from a uniform distribution on $[0,\theta]$, where $\theta$ was calibrated separately for each scenario to achieve target censoring rates of approximately 20%, 50%, and 80%.

For each simulated dataset, we constructed confidence intervals using two approaches. The first is the proposed nonparametric randomization-based method, obtained by inverting the log-rank test over a fine grid of candidate values of $\rho$. The second is a parametric approach based on a Weibull AFT model fit by maximum likelihood, with Wald-type confidence intervals constructed on the log scale and exponentiated to obtain intervals for $\rho$.

We considered sample sizes $n\in\{50,200\}$. For each combination of data-generating process, sample size, censoring rate, and true parameter value, we generated 3,000 simulated datasets. Performance under the null ($\rho=1$) was evaluated via Type I error, defined as the proportion of intervals failing to contain the true value. Performance under the alternative ($\rho=1.25$) was evaluated via coverage probability and median interval width.

We illustrate the proposed CIs using data from a randomized trial of recombinant human deoxyribonuclease I (rhDNase) for the treatment of cystic fibrosis [9]. Here, the primary endpoint is time to the first pulmonary exacerbation. The dataset is available as rhDNase in the survival package for R [8, 10]. Of the $n=647$ enrolled subjects, we use the $n=641$ with positive observed times (six subjects had an infection at enrollment and are excluded). Code 1 provides R code to prepare the dataset.

Of the 641 subjects, 323 were randomized to placebo ($Z=0$) and 318 to rhDNase ($Z=1$), with 138 and 103 events respectively. The log-rank test gives $p=0.0056$, providing evidence that rhDNase delays pulmonary exacerbation.

We apply the test-inversion procedure of Section 4 at the $\alpha=0.05$ level, searching over a grid of 1001 values of $c\in[-80,80]$ days for the additive shift. The additive point estimate is $50.1$ days with 95% CI $[17.1,\,73.0]$ days, indicating that rhDNase delayed the time to first exacerbation by between roughly two and ten weeks, with a point estimate of about 50 days. Code 2 includes an R function to estimate the additive effect.

To estimate the multiplicative factor, we apply the test-inversion procedure of Section 4 at the $\alpha=0.05$ level and search over a grid of 1001 values of $\rho$ on a log scale over $[e^{-3},e^{3}]$. The multiplicative point estimate $\hat{\rho}=1.55$ with 95% CI $[1.14,1.89]$ suggests that rhDNase increases the time to event by a factor of $1.55$. In other words, treatment is estimated to extend time to event by about 55% on a multiplicative time scale. Code 3 includes an R function to estimate the multiplicative effect.

As mentioned in Section 4.2, the adequacy of a constant-effect summary can be assessed visually by transforming the control arm’s Kaplan–Meier curve and checking whether it tracks the treatment arm. Figure 4 overlays two such transformations: the control curve shifted right by $\hat{c}=50$ days (additive) and the control curve with its time axis stretched by $\hat{\rho}=1.55$ (multiplicative), so that a step at time $t$ is replotted at $t+\hat{c}$ or $t\hat{\rho}$ respectively. The multiplicative rescaling tracks the treatment arm more closely across the follow-up period than the additive shift, supporting the use of $\hat{\rho}=1.55$ as the preferred summary of the treatment effect. Code 4 includes R code to create such a plot.

For ease of implementation, we have created a Shiny application, available at: https://lucy.shinyapps.io/survival_ci.

We describe two nonparametric confidence intervals for summarizing treatment effects for randomized trials with time-to-event endpoints, one for an additive shift and one for a multiplicative factor. Both are obtained by inverting the log-rank test under sharp null hypotheses of constant effects, inheriting the randomization-validity established by 4 without requiring any model for the time-to-event distribution.

A strength of the proposed approach is its conceptual alignment with existing nonparametric approaches used in practice. The log-rank test and Kaplan–Meier estimator are standard tools for survival analysis in randomized trials, yet neither provides a summary metric of the overall effect magnitude. The additive and multiplicative CIs proposed here are computed directly from the same log-rank statistic already often reported, require no additional distributional assumptions, and remain valid under the randomization distribution alone.

A related test-inversion approach was proposed by 5, who build on work by 6 to construct a confidence interval for the hazard ratio by inverting the partial-likelihood score test under the Cox proportional hazards model. Their method shares the property that the confidence interval excludes the null value if the log-rank test is significant, and it yields narrower and more accurate intervals than the standard Wald confidence interval. However, their procedure targets the hazard ratio, a parameter whose interpretation relies on the proportional hazards assumption, and validity depends on correct model specification. By contrast, the confidence intervals proposed here target an additive shift or multiplicative factor and require no assumptions for the event-time distribution. While both utilize the log-rank test, the distinction is that our approach requires fewer assumptions, deriving its validity from the randomization distribution alone.

Both parameters summarize the treatment effect by a single scalar and therefore are most meaningful when the effect is constant across time. In the presence of treatment effect heterogeneity, the Kaplan–Meier curves themselves along with measures such as restricted mean survival time may be more informative. Alternatively, the constant-effect assumption could be relaxed by considering subgroup analyses, should the subgroups be known.

A limitation is that the proposed CIs will generally be wider than their parametric counterparts when a parametric model is correctly specified. Because the log-rank statistic is rank-based, it discards the magnitude of the observed times and uses only their ordering. A correctly specified Weibull or log-normal AFT model, by contrast, exploits the full likelihood and will yield a more precise estimate of the acceleration factor $\rho$ under that model. This efficiency loss is a trade-off for robustness in that the nonparametric CIs are valid regardless of the event-time distribution. Practitioners who are confident in a parametric family may prefer the narrower intervals it affords.

Although our randomization-based justification requires assumptions on the censoring mechanism (specifically, noninformative i.i.d. censoring as stated in Assumption 2), it places no stochastic assumptions on the potential event times. In contrast, the classical asymptotic theory for the log-rank test treats the event times as stochastic and requires independent/noninformative censoring, while allowing more general censoring schemes than i.i.d. censoring [2]. Consequently, even in settings where the censoring assumptions underlying our randomization-based analysis fail, the proposed procedure may still admit a valid classical large-sample justification under an appropriate superpopulation model for the event times. In this sense, the procedure exhibits a form of robustness to modeling assumptions, since the randomization-based and classical asymptotic justifications rely on different sources of stochastic structure and impose assumptions on different components of the data-generating process.

In summary, the proposed confidence intervals offer a nonparametric complement to the log-rank test and Kaplan–Meier plots for summarizing time-to-event treatment effects in randomized trials. They are easy to compute using standard R tools, directly interpretable on clinically meaningful scales, and grounded in the same randomization-validity framework as the log-rank test itself. We hope the accompanying Shiny application makes them accessible to a broad audience of applied researchers.

Code to reproduce all figures and results can be found on Github: https://github.com/LucyMcGowan/2026-nonparametric-survival.