Individualized cost-effectiveness analysis (iCEA)

Devin Incerti

2017-12-01

Overview

Individualized cost-effectiveness analysis (iCEA) evaluates the cost-effectiveness of treatments at the individual (or subgroup) level. This has two major implications:

The hesim package help facilitate iCEA by providing a number of functions for analyzing subgroup level health and cost outcomes from simulation models that quantify parameter uncertainty using probabilistic sensitivity analysis (PSA). These functions take simulation output and generate measures commonly used for technology assessment including:

The rest of this document provides an overview of iCEA and how it can be conducted using hesim. The perspective is Bayesian in nature in that it is concerned with estimating the entire distribution of outcomes rather than just expected values (Baio 2012; Baio and Dawid 2015). It also stresses that both optimal treatments and the cost-effectiveness of those treatments vary across individuals (Basu and Meltzer 2007; Espinoza et al. 2014).

Net monetary benefits

Decision analysis provides a formal framework for making treatment decisions based on the utility that a therapy provides to a patient population. Decisions are typically made using a net benefit approach grounded in expected utility theory. The optimal treatment strategy is the one that maximizes expected NMBs where expected NMBs are calculated by averaging over the patient population and uncertain parameters \(\theta\). For a given subgroup \(g\) and parameter set \(\theta\), NMBs are computed as the difference between the monetized health gains from an intervention less costs, or,

\[ \begin{aligned} NMB_g(j,\theta) = e_{gj}\cdot k- c_{gj}, \end{aligned} \]

where \(e_{gj}\) and \(c_{gj}\) are measures of clinical effectiveness (e.g. QALYs) and costs in subgroup \(g\) using treatment \(j\) respectively, and \(k\) is a decision makers willingness to pay per unit of clinical effectiveness. The optimal treatment for a given subgroup is the one that maximizes expected NMBs,

\[ \begin{aligned} j^{*}_g = \text{argmax}_j E_{\theta} \left[NMB_g(j,\theta)\right]. \end{aligned} \]

In practice, new interventions are usually compared to a standard treatment often referred to as the comparator. In these cases, a new treatment in a given subgroup is preferred to the comparator if the expected incremental net monetary benefit (INMB) of the new treatment is positive; that is, treatment 1 is preferred to treatment 0 in subgroup \(g\) if \(E_\theta \left[INMB_g\right] > 0\) where the INMB in a particular subgroup is given by

\[ \begin{aligned} INMB_g(\theta) = NMB_g(j = 1, \theta) - NMB_g(j = 0, \theta). \end{aligned} \] Equivalently, treatment \(1\) is preferred to treatment \(0\) in subgroup \(g\) if the incremental cost-effectiveness ratio (ICER) is greater than the willingness to pay threshold \(k\),

\[ \begin{aligned} k > \frac{c_{g1} - c_{s0}}{e_{g1} - e_{g0}} = ICER_g. \end{aligned} \]

Probabilistic sensitivity analysis

Expected NMBs are expected values and ignore parameter uncertainty. This implies that NMBs are uncertain and that optimal treatment strategies may be selected incorrectly. This uncertainty can be quantified using PSA, which uses Bayesian and quasi-Bayesian techniques to estimate the distribution of NMBs given the distribution of the parameters for each treatment strategy

Since the joint distribution of the model parameters cannot be derived analytically (except in the simplest cases), the distribution of \(\theta\) is approximated by simulating the parameters from their joint posterior distribution and calculating relevant quantities of interest as a function of the simulated parameters. For each treatment strategy and subgroup, PSA therefore produces \(n\) random draws from the posterior distribution of clinical effectiveness and costs,

\[ \begin{aligned} e_{gj} &= [e_{gj}^1, e_{gj}^2, \dots, e_{gj}^n] \\ c_{gj} &= [c_{gj}^1, c_{gj}^2, \dots, c_{gj}^n]. \end{aligned} \]

Below we simulate costs and QALYs for three treatment strategies and two subgroups (in a real world analysis, this output would be derived from a detailed health-economic simulation model). Strategy 1 is the current standard of care; it is the cheapest therapy, but also the least efficacious. Strategies 2 and 3 are equally costly, but Strategy 2 is more effective in subgroup 1 while Strategy 3 is more effective in subgroup 2.

nsims <- 1000

# cost
c <- vector(mode = "list", length = 6)
names(c) <- c("Strategy 1, Grp 1", "Strategy 1, Grp 2", "Strategy 2, Grp 1",
              "Strategy 2, Grp 2", "Strategy 3, Grp 1", "Strategy 3, Grp 2")
c[[1]] <- rlnorm(nsims, 2, .1)
c[[2]] <- rlnorm(nsims, 2, .1)
c[[3]] <- rlnorm(nsims, 11, .15)
c[[4]] <- rlnorm(nsims, 11, .15)
c[[5]] <- rlnorm(nsims, 11, .15)
c[[6]] <- rlnorm(nsims, 11, .15)

# effectiveness
e <- c
e[[1]] <- rnorm(nsims, 8, .2)
e[[2]] <- rnorm(nsims, 8, .2)
e[[3]] <- rnorm(nsims, 10, .8)
e[[4]] <- rnorm(nsims, 10.5, .8)
e[[5]] <- rnorm(nsims, 8.5, .6)
e[[6]] <- rnorm(nsims, 11, .6)

# cost and effectiveness by strategy and simulation
library("data.table")
## Warning: package 'data.table' was built under R version 3.4.2
ce <- data.table(sim = rep(seq(nsims), length(e)),
                 strategy = rep(paste0("Strategy ", seq(1, 3)), 
                                each = nsims * 2),
                 grp = rep(rep(c("Group 1", "Group 2"),
                               each = nsims), 3),
                 cost = do.call("c", c), qalys = do.call("c", e))
head(ce)
##    sim   strategy     grp     cost    qalys
## 1:   1 Strategy 1 Group 1 8.451601 8.078954
## 2:   2 Strategy 1 Group 1 7.742073 7.940614
## 3:   3 Strategy 1 Group 1 7.811602 7.937523
## 4:   4 Strategy 1 Group 1 8.551299 8.116105
## 5:   5 Strategy 1 Group 1 6.434183 7.893628
## 6:   6 Strategy 1 Group 1 7.316293 8.282843

For any given willingness to pay \(k\), expected NMBs can be calculated by strategy, subgroup, and simulation number. For example, with \(k=150,000\), a reasonable estimate of the value of a life-year in the United States, Strategy 2 provides the highest expected NMBs in subgroup 2 while Strategy 3 provides the highest expected NMBs in subgroup 2.

ce <- ce[, nmb := 150000 * qalys - cost]
enmb <- ce[, .(enmb = mean(nmb)), by = c("strategy", "grp")]
enmb <- dcast(enmb, strategy ~ grp, value.var = "enmb")
print(enmb)
##      strategy Group 1 Group 2
## 1: Strategy 1 1200654 1200880
## 2: Strategy 2 1441759 1515136
## 3: Strategy 3 1213626 1590528

A number of measures have been proposed in the health economics literature to summarize the PSA. Below we describe the most common measures, which can be calculated using the functions icea and icea_pw. The icea function summarizes results by taking into account each treatment strategy in the analysis, while the function icea_pw summarizes “pairwise” results in which each treatment is compared to a comparator.

We can use the icea function to summarize results from our data.table object of simulated output for a range of willingness to pay values,

library("hesim")
ktop <- 200000
icea.dt <-  icea(ce, k = seq(0, ktop, 500), sim = "sim", strategy = "strategy",
                 grp = "grp", e = "qalys", c = "cost")

The most important input in icea is the data.table object (x) containing columns for simulation number (sim), treatment strategy (strategy), subgroup (grp), clinical effectiveness (e), and costs (c). Users specify the names of the relevant columns in their output table as strings. The other relevant parameter is \(k\), which is a range of willingness to pay values to use for estimating NMBs.

Likewise, we can use icea_pw to summarize the PSA when directly comparing the two treatment strategies (Strategy 2 and Strategy 3) to the comparator (Strategy 1).

icea.pw.dt <-  icea_pw(ce,  k = seq(0, ktop, 500), comparator = "Strategy 1",
                       sim = "sim", strategy = "strategy", e = "qalys", c = "cost")

The same inputs are used as in icea except users must specify the name of the comparator strategy.

Probability most cost-effective

A useful summary measure for quantifying uncertainty is the probability that each treatment strategy is the most cost effective. For a particular subgroup, this is estimated from simulation output as the proportion of simulation draws that each strategy has the highest NMB. For example, consider a random sample of 10 draws from the PSA simulation output and suppose \(k\) is again equal to \(150,000\).

library("knitr")
ce.nmb <- dcast(ce[sim %in% sample(1:nsims, 10) & grp == "Group 2"], 
                sim ~ strategy, value.var = "nmb")
setnames(ce.nmb, colnames(ce.nmb), c("sim", "nmb1", "nmb2", "nmb3"))
ce.nmb <- ce.nmb[, maxj := apply(ce.nmb[, .(nmb1, nmb2, nmb3)], 1, which.max)]
ce.nmb <- ce.nmb[, maxj := factor(maxj, levels = c(1, 2, 3))]
sim nmb1 nmb2 nmb3 maxj
125 1176690 1324295 1701511 3
207 1215923 1278078 1624231 3
237 1244828 1640593 1560367 2
293 1184355 1418157 1460112 3
324 1218831 1432620 1477070 3
375 1214018 1497974 1596836 3
516 1227468 1392205 1606803 3
527 1208966 1506985 1556830 3
843 1183681 1449984 1761407 3
907 1182825 1338191 1564894 3 
mce <- prop.table(table(ce.nmb$maxj))
print(mce)
## 
##   1   2   3 
## 0.0 0.1 0.9

In this example, treatments 1, 2, and 3 have the highest NMBs a fraction 0, 0.1, and 0.9 of the time respectively. The icea function performs this same calculations for a range of values of \(k\) and all nsims random draws of the simulation output. The output is a tidy data.table which facilitates plotting with ggplot.

library("ggplot2")
library("scales")
theme_set(theme_bw())
ggplot(icea.dt$mce, aes(x = k, y = prob, col = factor(strategy))) +
  geom_line() + facet_wrap(~grp) + xlab("Willingess to pay") +
  ylab("Probability most cost-effective") +
  scale_x_continuous(breaks = seq(0, ktop, 100000), label = comma) +
  theme(legend.position = "bottom") + scale_colour_discrete(name = "Strategy")

In group 1, Strategy 2 provides the greatest NMBs with high probability for almost all reasonable values of k. In group 2, the results are less certain, although Strategy 3 provides the greatest NMBs with a higher probability than Strategy 2.

Value of perfect information

One draw back of the previous measure is that it ignores the magnitude of cost or QALY gains. A measure which combines the probability of being most effective with the magnitude of the expected NMB is the expected value of perfect information (EVPI). Intuitively, the EVPI provides an estimate of the amount that a decision maker would be willing to pay to collect additional data and completely eliminate uncertainty. Mathematically, the EVPI is defined as the difference between the maximum expected NMB given perfect information and the maximum expected NMB given current information. In other words, we calculate the NMB for the optimal treatment strategy for each random draw of the parameters and compare that to the NMB for the treatment strategy that is optimal when averaging across all parameters. Mathematically, the EVPI for subgroup \(g\) is,

\[ \begin{aligned} EVPI_g &= E_\theta \left[max_j NMB_g(j, \theta)\right] - max_j E_\theta \left [ NMB_g(j, \theta)\right]. \\ \end{aligned} \]

To illustrate consider the same random sample of 10 draws from our simulation output used above.

strategymax.g2 <- which.max(enmb[[3]])
ce.nmb <- ce.nmb[, nmbpi := apply(ce.nmb[, .(nmb1, nmb2, nmb3)], 1, max)]
ce.nmb <- ce.nmb[, nmbci := ce.nmb[[strategymax.g2 + 1]]]
kable(ce.nmb, digits = 0, format = "html")
sim nmb1 nmb2 nmb3 maxj nmbpi nmbci
125 1176690 1324295 1701511 3 1701511 1701511
207 1215923 1278078 1624231 3 1624231 1624231
237 1244828 1640593 1560367 2 1640593 1560367
293 1184355 1418157 1460112 3 1460112 1460112
324 1218831 1432620 1477070 3 1477070 1477070
375 1214018 1497974 1596836 3 1596836 1596836
516 1227468 1392205 1606803 3 1606803 1606803
527 1208966 1506985 1556830 3 1556830 1556830
843 1183681 1449984 1761407 3 1761407 1761407
907 1182825 1338191 1564894 3 1564894 1564894


To calculate EVPI, we average NMBs given current information and NMBs given perfect information across simulation draws.

enmbpi <- mean(ce.nmb$nmbpi)
enmbci <- mean(ce.nmb$nmbci)
print(enmbpi)
## [1] 1599029
print(enmbci)
## [1] 1591006
print(enmbpi - enmbci)
## [1] 8022.606

The icea function performs this same calculation across all simulation draws from the PSA and for a number of values of willingness to pay values \(k\). A plot by group of the the EVPI for different values of \(k\) is shown below. The kinks in the plot represent values of \(k\) where the optimal strategy changes.

ggplot(icea.dt$evpi, aes(x = k, y = evpi)) +
  geom_line() + facet_wrap(~grp) + xlab("Willingess to pay") +
  ylab("Expected value of perfect information") +
  scale_x_continuous(breaks = seq(0, ktop, 100000), label = comma) +
  scale_y_continuous(label = scales::dollar) +
  theme(legend.position = "bottom") + scale_colour_discrete(name = "Strategy")

We might also be interested in aggregating across subgroups. In particular, the total EVPI can be calculated as a weighted average of the group specific EVPIs,

\[ \begin{aligned} totEVPI &= \sum_{g=1}^{G} w_g EVPI_g, \end{aligned} \] where \(w_g \in (0, 1)\) is a weight denoting that proportion of the population represented by subgroup \(g\) and \(\sum_{g=1}^{G} w_g = 1\).

w.dt <- data.table(grp = paste0("Group ", seq(1, 2)), w = c(0.25, .75))
evpi <- icea.dt$evpi
evpi <- merge(evpi, w.dt, by = "grp")
totevpi <- evpi[,lapply(.SD, weighted.mean, w = w),
                by = "k", .SDcols = c("evpi")]
ggplot(totevpi, aes(x = k, y = evpi)) +
  geom_line() + xlab("Willingess to pay") +
  ylab("Total EVPI") +
  scale_x_continuous(breaks = seq(0, ktop, 100000), label = comma) +
  scale_y_continuous(label = scales::dollar) +
  theme(legend.position = "bottom") + scale_colour_discrete(name = "Strategy")

Distribution of health and cost outcomes

icea produces means, 2.5% quantiles, and 97.5% quantiles for costs and efficacy.

print(icea.dt$summary)
##      strategy     grp    e_mean  e_lower   e_upper       c_mean
## 1: Strategy 1 Group 1  8.004409 7.623363  8.403913     7.400266
## 2: Strategy 2 Group 1 10.018823 8.365099 11.702954 61064.808952
## 3: Strategy 3 Group 1  8.492861 7.234508  9.671791 60302.696801
## 4: Strategy 1 Group 2  8.005920 7.626148  8.364747     7.451507
## 5: Strategy 2 Group 2 10.502892 8.938440 12.136374 60298.162381
## 6: Strategy 3 Group 2 11.007630 9.754234 12.204974 60616.754868
##         c_lower     c_upper
## 1:     6.047017     8.79667
## 2: 44857.970253 80525.52837
## 3: 44510.625487 80843.10278
## 4:     6.052045     9.03192
## 5: 45328.175368 78696.04760
## 6: 44603.260839 80516.77334

In addition, if the user would like to examine outcomes other than those specified this summary table, then they can generate a custom table of summary output. The custom table can contain any outcomes as long as they are specified in addition to the sim, strategy, e, and c columns in the posterior distribution data table. The default is to estimate means, the 2.5% quantile, and the 97.5% quantile for each variable, but any custom function can used. Below, we create a hypothetical variable for life-years and create a table summarizing our estimates of costs, QALYs and life-years. A custom function, identical to the default option, is entered into the function for illustrative purposes.

ce <- ce[, lys := qalys * 1.5]
cea.fun <- function(x) list(mean = mean(x), quant = quantile(x, c(.025, .975)))
icea.custom.dt <- icea(ce, k = seq(0, ktop, 500), sim = "sim", strategy = "strategy",
                       grp = "grp", e = "qalys", c = "cost",
                       custom_vars = c("cost", "lys", "qalys"), 
                       custom_fun = cea.fun)
icea.custom.dt$summary
##      strategy     grp    e_mean  e_lower   e_upper       c_mean
## 1: Strategy 1 Group 1  8.004409 7.623363  8.403913     7.400266
## 2: Strategy 2 Group 1 10.018823 8.365099 11.702954 61064.808952
## 3: Strategy 3 Group 1  8.492861 7.234508  9.671791 60302.696801
## 4: Strategy 1 Group 2  8.005920 7.626148  8.364747     7.451507
## 5: Strategy 2 Group 2 10.502892 8.938440 12.136374 60298.162381
## 6: Strategy 3 Group 2 11.007630 9.754234 12.204974 60616.754868
##         c_lower     c_upper
## 1:     6.047017     8.79667
## 2: 44857.970253 80525.52837
## 3: 44510.625487 80843.10278
## 4:     6.052045     9.03192
## 5: 45328.175368 78696.04760
## 6: 44603.260839 80516.77334
icea.custom.dt$custom.table
##      strategy     grp    cost.mean cost.quant.2.5% cost.quant.97.5%
## 1: Strategy 1 Group 1     7.400266        6.047017          8.79667
## 2: Strategy 2 Group 1 61064.808952    44857.970253      80525.52837
## 3: Strategy 3 Group 1 60302.696801    44510.625487      80843.10278
## 4: Strategy 1 Group 2     7.451507        6.052045          9.03192
## 5: Strategy 2 Group 2 60298.162381    45328.175368      78696.04760
## 6: Strategy 3 Group 2 60616.754868    44603.260839      80516.77334
##    lys.mean lys.quant.2.5% lys.quant.97.5% qalys.mean qalys.quant.2.5%
## 1: 12.00661       11.43505        12.60587   8.004409         7.623363
## 2: 15.02823       12.54765        17.55443  10.018823         8.365099
## 3: 12.73929       10.85176        14.50769   8.492861         7.234508
## 4: 12.00888       11.43922        12.54712   8.005920         7.626148
## 5: 15.75434       13.40766        18.20456  10.502892         8.938440
## 6: 16.51145       14.63135        18.30746  11.007630         9.754234
##    qalys.quant.97.5%
## 1:          8.403913
## 2:         11.702954
## 3:          9.671791
## 4:          8.364747
## 5:         12.136374
## 6:         12.204974

Cost-effectiveness plane

The cost-effectiveness plane plots the incremental effectiveness of a treatment strategy (relative to a comparator) against the incremental cost of the treatment strategy. The plot is useful because it demonstrates both the uncertainty and the magnitude of the estimates. Each point on the plot is from a particular random draw from the PSA.

Data for plotting a cost-effectiveness plane comes from the delta output generated from the icea_pw function, which, for each sampled parameter set and treatment strategy, estimates differences in costs, effectiveness, and any other variables specified in custom_varsrelative to the comparator. The dotted line in the plot is the willingness to pay line, with slope equal to the value of \(k\). For a given \(k\), points below the line are cost-effective while those above it are not.

head(icea.pw.dt$delta)
##    sim   strategy     grp        ie       ic
## 1:   1 Strategy 2 Group 1 0.1941625 68179.45
## 2:   2 Strategy 2 Group 1 2.4890219 70515.07
## 3:   3 Strategy 2 Group 1 2.5254637 63719.47
## 4:   4 Strategy 2 Group 1 1.7731137 74916.43
## 5:   5 Strategy 2 Group 1 1.9586024 65673.20
## 6:   6 Strategy 2 Group 1 3.1634414 48501.49
ylim <- max(icea.pw.dt$delta[, ic]) * 1.1
xlim <- ceiling(max(icea.pw.dt$delta[, ie]) * 1.1)
ggplot(icea.pw.dt$delta, aes(x = ie, y = ic, col = factor(strategy))) + 
  geom_jitter(size = .5) + facet_wrap(~grp) + 
  xlab("Incremental QALYs") + ylab("Incremental cost") +
  scale_y_continuous(label = dollar, limits = c(-ylim, ylim)) +
  scale_x_continuous(limits = c(-xlim, xlim), breaks = seq(-6, 6, 2)) +
  theme(legend.position = "bottom") + scale_colour_discrete(name = "Strategy") +
  geom_abline(slope = 150000, linetype = "dashed") +
  geom_hline(yintercept = 0) + geom_vline(xintercept = 0)

Cost-effectiveness acceptability curve (CEAC)

The cost-effectiveness acceptability curve (CEAC) is similar to the MCE plot. The difference is that the CEAC compares each strategy to a single comparator whereas the MCE plot considers all strategies simultaneously. Output to produce a CEAC is generated from the icea_pw function.

The plot shows that, in subgroup 1, Strategy 2 has larger NMBs than Strategy 1 with very high probability for reasonable values of \(k\). Strategy 3 also has higher NMBs than Strategy 1 with probability over 1/2 for values of \(k\) larger than 121,500. In group 2, both Strategy 2 and Strategy 3 have higher NMBs than Strategy 1 for almost all values of \(k\), although this probability is larger for Strategy 2 than Strategy 3 when \(k\) is smaller.

ggplot(icea.pw.dt$ceac, aes(x = k, y = prob, col = factor(strategy))) +
  geom_line() + facet_wrap(~grp) + xlab("Willingess to pay") +
  ylab("Probability most cost-effective") +
  scale_x_continuous(breaks = seq(0, ktop, 100000), label = comma) +
  theme(legend.position = "bottom") + scale_colour_discrete(name = "Strategy")

Credible intervals for incremental costs and effectiveness

Similar to the icea function, icea_pw produces means, 2.5% quantiles, and 97.5% quantiles for incremental costs and incremental efficacy. The table also contains the ICER, which is equal to expected incremental costs divided by the measure of expected incremental efficacy.

print(icea.pw.dt$summary)
##      strategy     grp   ie_mean   ie_lower ie_upper  ic_mean ic_lower
## 1: Strategy 2 Group 1 2.0144140  0.2920599 3.734516 61057.41 44850.73
## 2: Strategy 3 Group 1 0.4884516 -0.8117757 1.737028 60295.30 44503.54
## 3: Strategy 2 Group 2 2.4969724  0.8702740 4.289414 60290.71 45320.76
## 4: Strategy 3 Group 2 3.0017106  1.7278435 4.305857 60609.30 44596.14
##    ic_upper      icer
## 1: 80518.09  30310.26
## 2: 80836.63 123441.70
## 3: 78688.14  24145.53
## 4: 80508.57  20191.59

Again similar to the icea function, the user can also estimate incremental effects for outcome variables not included in the summary table.

Value of individualized care

The previous analyses allow NMBs and optimal treatment decisions to vary by subgroup. In contrast, most CEAs estimate the treatment, \(j^{*}\), that is optimal when averaging NMBs over the entire population. In particular, if the population is broken up into \(G\) distinct subgroups, the optimal treatment is given by,

\[ \begin{aligned} j^{*} = \text{argmax}_j \sum_{g=1}^{G} w_g E_{\theta}\left[NMB_g(j,\theta)\right]. \end{aligned} \]

Basu and Meltzer (2007) have shown that selecting subgroup specific treatments increases expected net benefits relative to this one-size fits all approach. They refer to additional net benefit as the expected value of individualized care (EPIC), which can be computed in terms of NMBs using the subgroup approach illustrated here as,

\[ \begin{aligned} \sum_{g=1}^G w_g E_{\theta}\left[NMB_g(j^{*}_s,\theta)\right] - \sum_{g=1}^G w_g E_{\theta}\left[NMB_g(j^{*},\theta)\right]. \end{aligned} \]

We can estimate the value of individualized care as follows:

ce <- merge(ce, w.dt, by = "grp")
totenmb <- ce[, .(totenmb = weighted.mean(nmb, w = w)), by = c("strategy")]
ptenmb.grp.max <- apply(as.matrix(enmb[, -1]), 2, max)
ptenmb.max <- sum(ptenmb.grp.max * w.dt$w)
tenmb.max <- max(totenmb$totenmb)
tnmb <- c(ptenmb.max, tenmb.max)
names(tnmb) <- c("Personalized total TENMB", "One-size fits all TENMB")
evic <- tnmb[1] - tnmb[2]
names(evic) <- "EVIC"
print(evic)
##    EVIC 
## 56544.1
print(evic/150000)
##      EVIC 
## 0.3769607

Our estimate of the EVIC is $56,544, or in terms of net health benefits, 0.377 QALYs.

References

Baio, Gianluca. 2012. Bayesian Methods in Health Economics. CRC Press.

Baio, Gianluca, and A Philip Dawid. 2015. “Probabilistic Sensitivity Analysis in Health Economics.” Statistical Methods in Medical Research 24 (6). SAGE Publications Sage UK: London, England: 615–34.

Basu, Anirban, and David Meltzer. 2007. “Value of Information on Preference Heterogeneity and Individualized Care.” Medical Decision Making 27 (2). Sage Publications: 112–27.

Espinoza, Manuel A, Andrea Manca, Karl Claxton, and Mark J Sculpher. 2014. “The Value of Heterogeneity for Cost-Effectiveness Subgroup Analysis: Conceptual Framework and Application.” Medical Decision Making 34 (8). Sage Publications Sage CA: Los Angeles, CA: 951–64.