Design of the DEFUSE3 Trial

Continuous Rankin Simulations

Here, we present the calculations for the initial design of the DEFUSE3 trial based on (Lai, Lavori, and Liao 2014) and (Lai, Lavori, and Tsang 2015). The trial parameters are fixed as follows.

library(ASSISTant)
##Fix randomization vector N, errors, eps
trialParameters <- list(N = c(200, 340, 476), type1Error = 0.025,
                        eps = 1/2, type2Error = 0.1)

The design parameters are the following for various scenarios.

designParameters <- list(
    nul0 = list(prevalence = rep(1/6, 6), mean = matrix(0, 2, 6),
                sd = matrix(1, 2, 6)),
    alt1 = list(prevalence = rep(1/6, 6), mean = rbind(rep(0, 6),
                                                       c(0.5, 0.4, 0.3, 0, 0, 0)),
                sd = matrix(1, 2, 6)),
    alt2 = list(prevalence = rep(1/6, 6), mean = rbind(rep(0, 6),
                                                     c(0.5, 0.5, 0, 0, 0, 0)),
                sd = matrix(1,2, 6)),
    alt3 = list(prevalence = rep(1/6, 6), mean = rbind(rep(0, 6), rep(0.36, 6)),
                sd = matrix(1,2, 6)),
    alt4 = list(prevalence = rep(1/6, 6), mean = rbind(rep(0, 6), rep(0.30, 6)),
                sd = matrix(1,2, 6)),
    alt5 = list(prevalence = rep(1/6, 6), mean = rbind(rep(0, 6),
                                                       c(0.4, 0.3, 0.2, 0, 0, 0)),
                sd = matrix(1,2, 6)),
    alt6 = list(prevalence = rep(1/6, 6), mean = rbind(rep(0, 6),
                                                       c(0.5, 0.5, 0.3, 0.3, 0.1, 0.1)),
                sd = matrix(1,2, 6))
)

The NULL Scenario

defuse3 <- DEFUSE3Design$new(trialParameters = trialParameters,
                             numberOfSimulations = 500,
                             designParameters = designParameters$nul0,
                             showProgress = FALSE)
print(defuse3)

## Design Parameters:
##  Number of Groups: 6
##  Prevalence:
## 
## |    Group1|    Group2|    Group3|    Group4|    Group5|    Group6|
## |---------:|---------:|---------:|---------:|---------:|---------:|
## | 0.1666667| 0.1666667| 0.1666667| 0.1666667| 0.1666667| 0.1666667|
## 
##  Using Discrete Rankin scores? FALSE
## 
##  Normal Rankin Distribution means (null row, alt. row):
## 
## 
## |     | Group1| Group2| Group3| Group4| Group5| Group6|
## |:----|------:|------:|------:|------:|------:|------:|
## |Null |      0|      0|      0|      0|      0|      0|
## |Alt  |      0|      0|      0|      0|      0|      0|
## 
##  Normal Rankin Distribution SDs (null row, alt. row):
## 
## 
## |     | Group1| Group2| Group3| Group4| Group5| Group6|
## |:----|------:|------:|------:|------:|------:|------:|
## |Null |      1|      1|      1|      1|      1|      1|
## |Alt  |      1|      1|      1|      1|      1|      1|
## 
## Trial Parameters:
## List of 6
##  $ N                 : num [1:3] 200 340 476
##  $ type1Error        : num 0.025
##  $ eps               : num 0.5
##  $ type2Error        : num 0.1
##  $ effectSize        : num 0.104
##  $ originalEffectSize: num 0.0858
## 
## Boundaries:
## 
## 
## |    btilde|        b|        c|
## |---------:|--------:|--------:|
## | -1.885434| 2.590764| 2.758372|

result <- defuse3$explore(numberOfSimulations = 500,
                          showProgress = FALSE,
                          rngSeed = 28912)
analysis <- defuse3$analyze(result)
print(defuse3$summary(analysis))

## P(Reject H0_ITT) = 0.002000; P(Reject H0_subgp) = 0.014000; P(Reject H0) = 0.016000
## P(Early stop for efficacy [futility]) = 0.008000 [0.662000]
## Mean [SD] Randomized N = 355.760000 [99.759743]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|      0.208|
## |         2|      0.462|
## |         3|      0.330|
## 
## Mean [SD] Lost N = 131.356000 [66.434089]
## Mean [SD] Analyzed N = 224.404000 [84.921093]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|      mean|        sd|
## |-------------:|-------------:|---------:|---------:|
## |             1|             1| 167.20359|  4.986172|
## |             1|             2| 132.76471|  6.999185|
## |             1|             3|  99.04082|  6.304625|
## |             1|             4|  67.68750|  5.735875|
## |             1|             5|  33.23636|  5.319300|
## |             2|             1| 280.93333|  7.362712|
## |             2|             2| 226.78947|  6.827875|
## |             2|             3| 171.25000| 11.804963|
## |             2|             4| 116.37500|  8.309633|
## |             2|             5|  55.08000|  5.589574|
## |             3|             1| 400.50000|  7.187953|
## |             3|             2| 304.75000| 18.300729|
## |             3|             3| 233.50000|  7.368853|
## |             3|             4| 158.33333| 10.016653|
## |             3|             5|  80.66667|  7.679173|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|     2|      0.004|
## |     2|     2|      0.004|
## |     3|     2|      0.004|
## |     4|     1|      0.002|
## |     6|     1|      0.002|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage|  G1| G2| G3| G4| G5|
## |-------------:|---:|--:|--:|--:|--:|
## |             1| 167| 68| 49| 48| 55|
## |             2|  15| 19|  8| 16| 25|
## |             3|   4|  4|  6|  3| 12|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |   0.984|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 0.9892473|           186|             2|
## | 0.9780220|            91|             2|
## | 0.9682540|            63|             2|
## | 0.9850746|            67|             1|
## | 1.0000000|            92|             0|
## | 0.0000000|             1|             1|
## NULL

The ALT1 Scenario

result1 <- defuse3$explore(numberOfSimulations = 500,
                           trueParameters = designParameters$alt1,
                           showProgress = FALSE,
                           rngSeed = 737218)
analysis1 <- defuse3$analyze(result1)
print(defuse3$summary(analysis1))

## P(Reject H0_ITT) = 0.284000; P(Reject H0_subgp) = 0.558000; P(Reject H0) = 0.842000
## P(Early stop for efficacy [futility]) = 0.436000 [0.018000]
## Mean [SD] Randomized N = 395.776000 [97.950203]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|      0.132|
## |         2|      0.322|
## |         3|      0.546|
## 
## Mean [SD] Lost N = 135.228000 [109.645375]
## Mean [SD] Analyzed N = 260.548000 [102.724449]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|      mean|        sd|
## |-------------:|-------------:|---------:|---------:|
## |             1|             1| 167.60417|  4.569741|
## |             1|             2| 134.41176|  7.912850|
## |             1|             3| 102.96875|  5.474188|
## |             1|             4|  68.50000|  3.728270|
## |             1|             5|  32.28571|  3.352327|
## |             2|             1| 281.89474|  5.404676|
## |             2|             2| 226.16667|  9.318829|
## |             2|             3| 168.87879|  8.513803|
## |             2|             4| 115.60000|  7.604093|
## |             2|             5|  57.00000|  7.958224|
## |             3|             1| 396.46154|  6.678515|
## |             3|             2| 319.29630|  7.700224|
## |             3|             3| 236.03125| 11.188101|
## |             3|             4| 162.05556| 10.032138|
## |             3|             5|  80.42105|  8.408433|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|    69|      0.138|
## |     2|    72|      0.144|
## |     3|   105|      0.210|
## |     4|    23|      0.046|
## |     5|    10|      0.020|
## |     6|   142|      0.284|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1| 48| 34| 32|  6|  7|
## |             2| 19| 24| 33| 10|  4|
## |             3| 13| 27| 64| 18| 19|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |   0.158|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 0.1375000|            80|            69|
## | 0.1529412|            85|            72|
## | 0.1860465|           129|           105|
## | 0.3235294|            34|            23|
## | 0.6666667|            30|            10|
## | 0.0000000|           142|           142|
## NULL

The ALT2 Scenario

result2 <- defuse3$explore(numberOfSimulations = 500,
                           trueParameters = designParameters$alt2,
                           showProgress = FALSE,
                          rngSeed = 928812)
analysis2 <- defuse3$analyze(result2)
print(defuse3$summary(analysis2))

## P(Reject H0_ITT) = 0.226000; P(Reject H0_subgp) = 0.620000; P(Reject H0) = 0.846000
## P(Early stop for efficacy [futility]) = 0.438000 [0.014000]
## Mean [SD] Randomized N = 402.488000 [89.064712]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|      0.086|
## |         2|      0.366|
## |         3|      0.548|
## 
## Mean [SD] Lost N = 165.346000 [118.444978]
## Mean [SD] Analyzed N = 237.142000 [103.804569]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|      mean|        sd|
## |-------------:|-------------:|---------:|---------:|
## |             1|             1| 166.85294|  5.105745|
## |             1|             2| 134.47674|  6.722616|
## |             1|             3| 100.66667|  9.232448|
## |             1|             4|  67.75000|  4.267820|
## |             1|             5|  32.00000|  2.943920|
## |             2|             1| 282.35294|  4.872643|
## |             2|             2| 227.91935|  8.218927|
## |             2|             3| 170.80000|  7.450578|
## |             2|             4| 117.00000|  6.164414|
## |             2|             5|  60.66667|  8.386497|
## |             3|             1| 400.30769|  8.606050|
## |             3|             2| 316.87368| 10.026817|
## |             3|             3| 242.84211|  7.581140|
## |             3|             4| 165.00000|  8.194074|
## |             3|             5|  80.55556|  5.052502|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|    54|      0.108|
## |     2|   216|      0.432|
## |     3|    27|      0.054|
## |     4|    11|      0.022|
## |     5|     2|      0.004|
## |     6|   113|      0.226|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1| 34| 86| 15|  8|  4|
## |             2| 17| 62| 10|  4|  3|
## |             3| 13| 95| 19|  8|  9|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |   0.154|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 0.1562500|            64|            54|
## | 0.1111111|           243|           216|
## | 0.3863636|            44|            27|
## | 0.4500000|            20|            11|
## | 0.8750000|            16|             2|
## | 0.0000000|           113|           113|
## NULL

Discrete Rankin Simulations

The Discretized Scenarios

The discretized scenarios are designed to generally mimic the trends above in the alternatives. However, we have a problem: we cannot simulatenously match the mean and sd of the alternatives above. (Actually, we can, but not with Rankin scores 0 through 6. The software can easily be modified to generate discrete values where the values are 0 to 6 divided by the standard deviation of the respective distribution, for example.)

Also in future versions, I need to allow for more general support values for the scores, not just 0 through 6. Easy to do, but not done yet.

Some types of distributions:

null.uniform <- rep(1, 7L) ## uniform on 7 support points
hourglass <- c(1, 2, 2, 1, 2, 2, 1)
inverted.hourglass <- c(2, 1, 1, 2, 1, 1, 2)
bottom.heavy <- c(2, 2, 2, 1, 1, 1, 1)
bottom.heavier <- c(3, 3, 2, 2, 1, 1, 1)
bottom.loaded <- c(4, 4, 3, 3, 2, 1, 1)
top.heavy <- c(1, 1, 1, 1, 2, 2, 2)
top.heavier <- c(1, 1, 1, 2, 2, 3, 3)
top.loaded <- c(1, 1, 2, 3, 3, 4, 4)

It is instructive to see what the means and standard deviations are.

knitr::kable(
           sapply(list(null = null.uniform,
                       hourglass = hourglass,
                       inv.hourglass = inverted.hourglass,
                       bot.heavy = bottom.heavy,
                       bot.heavier = bottom.heavier,
                       bot.loaded = bottom.loaded,
                       top.heavy = top.heavy,
                       top.heavier = top.heavier,
                       top.loaded = top.loaded),
                  computeMeanAndSD)
       )

	null	hourglass	inv.hourglass	bot.heavy	bot.heavier	bot.loaded	top.heavy	top.heavier	top.loaded
mean	3	3.000000	3.000000	2.400000	2.153846	2.111111	3.600000	3.846154	3.888889
sd	2	1.858641	2.144761	1.959592	1.874778	1.760331	1.959592	1.874778	1.760331

With this in mind, we can reel off some runs. Phil, you mentioned you wanted J = 2, which I adhere to, below.

designParameters <- list(
    nul0 = list(prevalence = rep(1, 2),
                ctlDist = null.uniform,
                trtDist = cbind(null.uniform,
                                null.uniform)),
    alt1 = list(prevalence = rep(1, 2), 
                ctlDist = null.uniform,
                trtDist = cbind(top.loaded,
                                null.uniform)),
    alt2 = list(prevalence = rep(1, 2), 
                ctlDist = null.uniform,
                trtDist = cbind(null.uniform,
                                top.loaded))
)

The NULL Scenario

discDefuse3 <- DEFUSE3Design$new(trialParameters = trialParameters,
                                 numberOfSimulations = 5000,
                                 discreteData = TRUE,
                                 designParameters = designParameters$nul0,
                                 showProgress = FALSE)
print(discDefuse3)

## Design Parameters:
##  Number of Groups: 2
##  Prevalence:
## 
## | Group1| Group2|
## |------:|------:|
## |    0.5|    0.5|
## 
##  Using Discrete Rankin scores? TRUE
## 
##  Null Rankin Distribution:
## 
## |   |    Group1|    Group2|
## |:--|---------:|---------:|
## |0  | 0.1428571| 0.1428571|
## |1  | 0.1428571| 0.1428571|
## |2  | 0.1428571| 0.1428571|
## |3  | 0.1428571| 0.1428571|
## |4  | 0.1428571| 0.1428571|
## |5  | 0.1428571| 0.1428571|
## |6  | 0.1428571| 0.1428571|
##  Null Mean and SD
## 
## |     | Group1| Group2|
## |:----|------:|------:|
## |mean |      3|      3|
## |sd   |      2|      2|
##  Alternative Rankin Distribution:
## 
## 
## |   |    Group1|    Group2|
## |:--|---------:|---------:|
## |0  | 0.1428571| 0.1428571|
## |1  | 0.1428571| 0.1428571|
## |2  | 0.1428571| 0.1428571|
## |3  | 0.1428571| 0.1428571|
## |4  | 0.1428571| 0.1428571|
## |5  | 0.1428571| 0.1428571|
## |6  | 0.1428571| 0.1428571|
##  Alternative Mean and SD
## 
## |     | Group1| Group2|
## |:----|------:|------:|
## |mean |      3|      3|
## |sd   |      2|      2|
## 
## Trial Parameters:
## List of 6
##  $ N                 : num [1:3] 200 340 476
##  $ type1Error        : num 0.025
##  $ eps               : num 0.5
##  $ type2Error        : num 0.1
##  $ effectSize        : num 0.106
##  $ originalEffectSize: num 0.0858
## 
## Boundaries:
## 
## 
## |    btilde|       b|        c|
## |---------:|-------:|--------:|
## | -1.875887| 2.48671| 2.467753|

result <- discDefuse3$explore(numberOfSimulations = 50,
                              showProgress = FALSE,
                              rngSeed = 3783)
analysis <- discDefuse3$analyze(result)
print(discDefuse3$summary(analysis))

## P(Reject H0_ITT) = 0.000000; P(Reject H0_subgp) = 0.020000; P(Reject H0) = 0.020000
## P(Early stop for efficacy [futility]) = 0.000000 [0.800000]
## Mean [SD] Randomized N = 305.600000 [106.616995]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.44|
## |         2|       0.36|
## |         3|       0.20|
## 
## Mean [SD] Lost N = 116.780000 [38.398709]
## Mean [SD] Analyzed N = 188.820000 [86.822054]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|       sd|
## |-------------:|-------------:|--------:|--------:|
## |             1|             1|  99.5000| 7.452413|
## |             2|             1| 165.4286| 9.589180|
## |             3|             1| 233.6667| 4.041452|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|     1|       0.02|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1|
## |-------------:|--:|
## |             1| 40|
## |             2|  7|
## |             3|  3|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |       1|         1|
## 
## P(theta_test is in the confidence interval)
## 
## 
## | coverage| selectedCount| rejectedCount|
## |--------:|-------------:|-------------:|
## |        1|            50|             1|
## NULL

The ALT1 Scenario

result1 <- discDefuse3$explore(numberOfSimulations = 50,
                               trueParameters = designParameters$alt1,
                               showProgress = FALSE,
                               rngSeed = 28912)
analysis1 <- discDefuse3$analyze(result1)
print(discDefuse3$summary(analysis1))

## P(Reject H0_ITT) = 0.520000; P(Reject H0_subgp) = 0.400000; P(Reject H0) = 0.920000
## P(Early stop for efficacy [futility]) = 0.560000 [0.000000]
## Mean [SD] Randomized N = 374.640000 [103.343643]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.18|
## |         2|       0.38|
## |         3|       0.44|
## 
## Mean [SD] Lost N = 83.040000 [97.380164]
## Mean [SD] Analyzed N = 291.600000 [89.833860]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|        sd|
## |-------------:|-------------:|--------:|---------:|
## |             1|             1| 100.5556|  9.951270|
## |             2|             1| 169.2500|  9.673848|
## |             3|             1| 233.6364| 15.298841|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|    20|       0.40|
## |     2|    26|       0.52|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1|
## |-------------:|--:|
## |             1|  9|
## |             2|  4|
## |             3| 11|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |       1|         1|
## 
## P(theta_test is in the confidence interval)
## 
## 
## | coverage| selectedCount| rejectedCount|
## |--------:|-------------:|-------------:|
## |        1|            24|            20|
## |        1|            26|            26|
## NULL

The ALT2 Scenario

result2 <- discDefuse3$explore(numberOfSimulations = 50,
                               trueParameters = designParameters$alt2,
                               showProgress = FALSE,
                               rngSeed = 931)
analysis2 <- discDefuse3$analyze(result2)
print(discDefuse3$summary(analysis2))

## P(Reject H0_ITT) = 0.560000; P(Reject H0_subgp) = 0.000000; P(Reject H0) = 0.560000
## P(Early stop for efficacy [futility]) = 0.380000 [0.340000]
## Mean [SD] Randomized N = 336.080000 [106.155035]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.30|
## |         2|       0.42|
## |         3|       0.28|
## 
## Mean [SD] Lost N = 68.340000 [86.598442]
## Mean [SD] Analyzed N = 267.740000 [129.424757]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|  mean|       sd|
## |-------------:|-------------:|-----:|--------:|
## |             1|             1| 100.8| 8.011103|
## |             2|             1| 171.0| 5.567764|
## |             3|             1| 242.4| 5.856620|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     2|    28|       0.56|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1|
## |-------------:|--:|
## |             1| 10|
## |             2|  7|
## |             3|  5|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |       1|         1|
## 
## P(theta_test is in the confidence interval)
## 
## 
## | coverage| selectedCount| rejectedCount|
## |--------:|-------------:|-------------:|
## |        1|            22|             0|
## |        1|            28|            28|
## NULL

References

Lai, Tze Leung, Philip W. Lavori, and Olivia Yueh-Wen Liao. 2014. “Adaptive Choice of Patient Subgroup for Comparing Two Treatments.” Contemporary Clinical Trials 39 (2): 191–200. https://doi.org/10.1016/j.cct.2014.09.001.

Lai, Tze Leung, Philip W. Lavori, and Ka Wai Tsang. 2015. “Adaptive Design of Confirmatory Trials: Advances and Challenges.” Contemporary Clinical Trials 45, Part A: 93–102. https://doi.org/10.1016/j.cct.2015.06.007.