Trinormal based ROC test

A statistical test function to assess three-class ROC data. It is possible to investigate a single classifier or make a comparison of two independent / correlated classifiers.

trinROC.test(
  x1,
  y1,
  z1,
  x2 = 0,
  y2 = 0,
  z2 = 0,
  dat = NULL,
  paired = FALSE,
  conf.level = 0.95
)

Arguments

x1, y1, z1: (non-empty) numeric vectors of data from the healthy, intermediate and diseased class from Classifier 1.
x2, y2, z2: numeric vectors of data from the healthy, intermediate and diseased class from Classifier 2.
dat: a data frame of the following structure: The first column represents a factor with three levels, containing the true class membership of each measurement. The levels are ordered according to the convention of higher values for more severe disease status. The second column contains all measurements obtained from Classifier 1 (in the case of single marker assessment). In the case of comparison of two markers, column three contains the measurementss from the Classifier.
paired: a logical indicating whether data arose from a paired setting. If TRUE, each class must have equal sample size for both classifiers.
conf.level: confidence level of the interval. A numeric value between (0,1) yielding the significance level $\alpha=1-$conf.level.

Value

A list of class "htest" containing the following components:

statistic: the value of the chi-squared statistic.
parameter: the degrees of freedom for the chi-squared statistic.
p.value: the p-value for the test.
conf.int: a confidence interval for the test.
estimate: a data frame containing the estimated VUS and parameters a, b, c and d from Classifier 1 and Classifier 2 (if specified).
null.value: a character expressing the null hypothesis.
alternative: a character string describing the alternative hypothesis.
method: a character string indicating what type of trinormal based ROC test was performed.
data.name: a character string giving the names of the data.
CovMat: the covariance matrix of the chi-squared statistic.
Summary: a data frame representing the number of NA's as well as the means and the standard deviations per class.

Details

The trinormal ROC model is a parametric model in three-class ROC analysis. It is based on normality in each of the trhee classes D_- (healthy), D_0 (intermediate) and D_+ (diseased) with denoted distributions $N(\mu_-,\sigma_-^2)$, $N(\mu_0,\sigma_0^2)$ and $N(\mu_+,\sigma_+^2)$. A classifier of a trinormal ROC model classifies individuals into one of the three ordered classes based on two cut-off points $c_- < c_+$. We define $t_-=F_-(c_-)$ and $t_+ =1-F_+(c_+)=G_+(c_+)$. Now, the ROC surface can be written as

$$ROCs(t_-,t_+) = \Phi \left(\frac{\Phi^{-1} (1-t_+) +d}{c} \right) - \Phi \left(\frac{\Phi^{-1} (t_-)+b}{a} \right)$$

whith parameters a, b, c and c given by $a = \frac{\hat{\sigma}_0}{\hat{\sigma}_-}, b = \frac{ \hat{\mu}_- - \hat{\mu}_0}{\hat{\sigma}_-}, c = \frac{\hat{\sigma}_0}{\hat{\sigma}_+}, d = \frac{ \hat{\mu}_+ - \hat{\mu}_0}{\hat{\sigma}_+} $. It is a surface in the unit cube that plots the probability of a measurement to get assigned to the intermediate class as the two thresholds $c_-,c_+$ are varying.

Based on the reference standard, the trinormal based ROC test can be used to assess the discriminatory power of such classifiers. It distinguishes between single classifier assessment, where a classifier is compared to some hypothetical distributions in the classes, and comparison between two classifiers. The latter case tests for equality between the parameters a, b, c and d of the ROC curves. The data can arise in a unpaired or paired setting. If paired is TRUE, a correlation is introduced which has to be taken into account. Therefore the sets of the two classifiers have to have classwise equal size. The data can be input as the data frame dat or as single vectors x1, y1, z1, ....

As the Chi-squared test is by definition a one-sided test, the variable alternative cannot be specified in this test. For this 'goodness of fit' test, we assume the parameters $a_1, \dots , d_1$ and $a_2, \dots , d_2$ to have a pairwise equivalent normal distribution (in large sample sets).

References

Noll, S., Furrer, R., Reiser, B. and Nakas, C. T. (2019). Inference in ROC surface analysis via a trinormal model-based testing approach. Stat, 8(1), e249.

Examples

data(cancer)
data(krebs)

# investigate a single marker:
trinROC.test(dat = cancer[,c(1,3)])
#> 
#> 	Trinormal based ROC test for single classifier assessment
#> 
#> data:  healthy  intermediate  diseased  of  Class2
#> Chi-Squared test = 49.989, df = 4, p-value = 3.629e-10
#> alternative hypothesis: true a1-a2, b1-b1, c1-c2 and d1-d2 is not equal to 0
#> sample estimates:
#>              VUS        a         b         c         d
#> Class2 0.4328247 1.798492 -1.283656 0.8268255 0.5835495
#> 
trinROC.test(dat = krebs[,c(1,5)])
#> 
#> 	Trinormal based ROC test for single classifier assessment
#> 
#> data:  healthy  intermediate  diseased  of  Fac4
#> Chi-Squared test = 16.926, df = 4, p-value = 0.001998
#> alternative hypothesis: true a1-a2, b1-b1, c1-c2 and d1-d2 is not equal to 0
#> sample estimates:
#>            VUS       a          b         c         d
#> Fac4 0.3274583 1.39355 -0.3395081 0.7770584 0.6141944
#> 

# result is equal to:
x1 <- with(cancer, cancer[trueClass=="healthy", 3])
y1 <- with(cancer, cancer[trueClass=="intermediate", 3])
z1 <- with(cancer, cancer[trueClass=="diseased", 3])
trinROC.test(x1, y1, z1)
#> 
#> 	Trinormal based ROC test for single classifier assessment
#> 
#> data:  x1 y1 and z1
#> Chi-Squared test = 49.989, df = 4, p-value = 3.629e-10
#> alternative hypothesis: true a1-a2, b1-b1, c1-c2 and d1-d2 is not equal to 0
#> sample estimates:
#>                    VUS        a         b         c         d
#> Classifier1: 0.4328247 1.798492 -1.283656 0.8268255 0.5835495
#> 

# comparison of marker 2 and 6:
trinROC.test(dat = cancer[,c(1,3,5)], paired = TRUE)
#> 
#> 	Trinormal based ROC test for comparison of paired ROC surfaces
#> 
#> data:  healthy intermediate diseased of Class2  and  healthy intermediate diseased of Class4
#> Chi-Squared test = 11.451, df = 4, p-value = 0.02193
#> alternative hypothesis: true a1-a2, b1-b1, c1-c2 and d1-d2 is not equal to 0
#> sample estimates:
#>              VUS        a         b         c         d
#> Class2 0.4328247 1.798492 -1.283656 0.8268255 0.5835495
#> Class4 0.3560958 1.347723 -1.229033 1.0790503 0.2033175
#> 
trinROC.test(dat = cancer[,c(1,3,5)], paired = FALSE)
#> 
#> 	Trinormal based ROC test for comparison of two independent classifiers
#> 
#> data:  healthy intermediate diseased of Class2  and  healthy intermediate diseased of Class4
#> Chi-Squared test = 7.1352, df = 4, p-value = 0.1289
#> alternative hypothesis: true a1-a2, b1-b1, c1-c2 and d1-d2 is not equal to 0
#> sample estimates:
#>              VUS        a         b         c         d
#> Class2 0.4328247 1.798492 -1.283656 0.8268255 0.5835495
#> Class4 0.3560958 1.347723 -1.229033 1.0790503 0.2033175
#> 

# result is equal to:
x2 <- with(cancer, cancer[trueClass=="healthy", 5])
y2 <- with(cancer, cancer[trueClass=="intermediate", 5])
z2 <- with(cancer, cancer[trueClass=="diseased", 5])
trinROC.test(x1, y1, z1, x2, y2, z2, paired = TRUE)
#> 
#> 	Trinormal based ROC test for comparison of paired ROC surfaces
#> 
#> data:  x1 y1 z1 and x2 y2 z2
#> Chi-Squared test = 11.451, df = 4, p-value = 0.02193
#> alternative hypothesis: true a1-a2, b1-b1, c1-c2 and d1-d2 is not equal to 0
#> sample estimates:
#>                    VUS        a         b         c         d
#> Classifier1: 0.4328247 1.798492 -1.283656 0.8268255 0.5835495
#> Classifier2: 0.3560958 1.347723 -1.229033 1.0790503 0.2033175
#>

Arguments

Value

Details

References

See also

Examples