trinVUS.test.RdA statistical test function to assess three-class ROC data. It can be used for assessment of a single classifier or comparison of two independent / correlated classifiers, using the statistical test developed by Xiong et al. (2007).
trinVUS.test(
x1,
y1,
z1,
x2 = 0,
y2 = 0,
z2 = 0,
dat = NULL,
paired = FALSE,
conf.level = 0.95,
alternative = c("two.sided", "less", "greater")
)non-empty numeric vectors of data from the healthy, intermediate and diseased class from Classifier 1.
numeric vectors of data from the healthy, intermediate and diseased class from Classifier 2, only needed in a comparison of two classifiers.
a data frame of the following structure: The first column represents a factor with three levels, containing the true class membership of each measurement. The levels are ordered according to the convention of higher values for more severe disease status. The second column contains all measurements obtained from Classifier 1 (in the case of single marker assessment). In the case of comparison of two markers, column three contains the measurementss from the Classifier.
logical; indicating whether data arose from a paired setting.
If TRUE, each class must have equal sample size for both
classifiers.
confidence level of the interval. A numeric value between (0,1) yielding the significance level \(\alpha=1-\code{conf.level}\).
character string specifying the alternative hypothesis,
must be one of "two.sided" (default), "greater" or "less". You can specify
just the initial letter. For two sided test, notice \(H0: Z = (VUS_1-VUS_2) /
(Var(VUS_1)+Var(VUS_2)-2Cov(VUS_1,VUS_2))^{0.5}\).
A list of class "htest" containing the following components:
the value of the Z-statistic.
the p-value for the test.
a confidence interval for the test.
a data frame containing the estimated VUS from Classifier 1 and Classifier 2 (if specified).
a character expressing the null hypothesis.
a character string describing the alternative hypothesis.
a character string indicating what type of trinormal VUS test was performed.
a character string giving the names of the data.
a data frame representing the number of NA's as well as the means and the standard deviations per class.
the covariance matrix of the VUS.
Based on the reference standard, this trinormal VUS test assesses the
discriminatory power of classifiers by comparing the volumes under the ROC
surfaces (VUS). It distinguishes between single classifier assessment,
where a classifier is compared to the chance plane with VUS=1/6, and
comparison between two classifiers. The latter case tests the equality
between VUS_1 and VUS_2. The data can arise in a unpaired or paired
setting. If paired is TRUE, a correlation is introduced which
has to be taken into account. Therefore the sets of the two classifiers
have to have classwise equal size. The data can be input as the data
frame dat or as single vectors x1, y1, z1, ....
Xiong, C., Van Belle, G. Miller J. P., Morris, J. C. (2006). Measuring and estimating diagnostic accuracy when there are three ordinal diagnostic groups. Statistics in Medicine, 25(7), 1251–1273.
Xiong, C., van Belle, G., Miller, J. P., Yan, Y., Gao, F., Yu, K., and Morris, J. C. (2007). A parametric comparison of diagnostic accuracy with three ordinal diagnostic groups. Biometrical Journal, 49(5), 682–693. doi: 10.1002/bimj.200610359 .
data(cancer)
data(krebs)
# investigate a single marker:
trinVUS.test(dat = cancer[,c(1,3)])
#>
#> Trinormal VUS test for single classifier assessment
#>
#> data: healthy intermediate diseased of Class2
#> Z-stat = 4.7208, p-value = 2.349e-06
#> alternative hypothesis: true Difference in VUS is not equal to 0
#> sample estimates:
#> VUS of Class2
#> 0.4328247
#>
trinVUS.test(dat = krebs[,c(1,5)])
#>
#> Trinormal VUS test for single classifier assessment
#>
#> data: healthy intermediate diseased of Fac4
#> Z-stat = 2.2327, p-value = 0.02557
#> alternative hypothesis: true Difference in VUS is not equal to 0
#> sample estimates:
#> VUS of Fac4
#> 0.3274583
#>
# result is equal to:
x1 <- with(cancer, cancer[trueClass=="healthy", 3])
y1 <- with(cancer, cancer[trueClass=="intermediate", 3])
z1 <- with(cancer, cancer[trueClass=="diseased", 3])
trinVUS.test(x1, y1, z1)
#>
#> Trinormal VUS test for single classifier assessment
#>
#> data: x1 y1 and z1
#> Z-stat = 4.7208, p-value = 2.349e-06
#> alternative hypothesis: true Difference in VUS is not equal to 0
#> sample estimates:
#> VUS of Classifier 1
#> 0.4328247
#>
# comparison of marker 2 and 6:
trinVUS.test(dat = cancer[,c(1,3,5)], paired = TRUE)
#>
#> Trinormal VUS test for comparison of paired ROC data
#>
#> data: healthy intermediate diseased of Class2 and healthy intermediate diseased of Class4
#> Z-stat = 1.3584, p-value = 0.1743
#> alternative hypothesis: true Difference in VUS is not equal to 0
#> sample estimates:
#> VUS of Class2 VUS of Class4
#> 0.4328247 0.3560958
#>
trinVUS.test(dat = cancer[,c(1,3,5)], paired = FALSE)
#>
#> Trinormal VUS test for comparison of two independent classifiers
#>
#> data: healthy intermediate diseased of Class2 and healthy intermediate diseased of Class4
#> Z-stat = 0.99489, p-value = 0.3198
#> alternative hypothesis: true Difference in VUS is not equal to 0
#> sample estimates:
#> VUS of Class2 VUS of Class4
#> 0.4328247 0.3560958
#>
# result is equal to:
x2 <- with(cancer, cancer[trueClass=="healthy", 5])
y2 <- with(cancer, cancer[trueClass=="intermediate", 5])
z2 <- with(cancer, cancer[trueClass=="diseased", 5])
trinVUS.test(x1, y1, z1, x2, y2, z2, paired = TRUE)
#>
#> Trinormal VUS test for comparison of paired ROC data
#>
#> data: x1 y1 z1 and x2 y2 z2
#> Z-stat = 1.3584, p-value = 0.1743
#> alternative hypothesis: true Difference in VUS is not equal to 0
#> sample estimates:
#> VUS of Classifier 1 VUS of Classifier 2
#> 0.4328247 0.3560958
#>