Testing the difference between two Kolmogorov-Smirnov values in the context of receiver operating characteristic curves
The maximum vertical distance between a receiver operating characteristic (ROC) curve and its chance diagonal is a common measure of effectiveness of the classifier that gives rise to this curve. This measure is known to be equivalent to a two-sample Kolmogorov-Smirnov statistic; so
the absolute difference D between two such statistics is often used informally as a measure of difference between the corresponding classifiers. A significance test of D is of great practical interest, but the available Kolmogorov-Smirnov distribution theory precludes easy analytical construction
of such a significance test. We, therefore, propose a Monte Carlo procedure for conducting the test, using the binormal model for the underlying ROC curves. We provide Splus/R routines for the computation, tabulate the results for a number of illustrative cases, apply the methods to some practical
examples and discuss some implications.
No Reference information available - sign in for access.
No Citation information available - sign in for access.
No Supplementary Data.
No Article Media
Monte Carlo methods;
conditional and marginal inference;
Document Type: Research Article
School of Engineering, Computing and Mathematics, University of Exeter, Exeter, UK,Department of Mathematics, Imperial College of Science, Technology and Medicine, UK
Department of Mathematics, Imperial College of Science, Technology and Medicine, UK,Institute for Mathematical Sciences, Imperial College of Science, Technology and Medicine, UK
Publication date: March 1, 2011