Testing the difference between two Kolmogorov-Smirnov values in the context of receiver operating characteristic curves
The maximum vertical distance between a receiver operating characteristic (ROC) curve and its chance diagonal is a common measure of effectiveness of the classifier that gives rise to this curve. This measure is known to be equivalent to a two-sample Kolmogorov-Smirnov statistic; so
the absolute difference D between two such statistics is often used informally as a measure of difference between the corresponding classifiers. A significance test of D is of great practical interest, but the available Kolmogorov-Smirnov distribution theory precludes easy analytical construction
of such a significance test. We, therefore, propose a Monte Carlo procedure for conducting the test, using the binormal model for the underlying ROC curves. We provide Splus/R routines for the computation, tabulate the results for a number of illustrative cases, apply the methods to some practical
examples and discuss some implications.
Keywords: Monte Carlo methods; Splus/R routines; binormal model; conditional and marginal inference; paired samples
Document Type: Research Article
Affiliations: 1: School of Engineering, Computing and Mathematics, University of Exeter, Exeter, UK,Department of Mathematics, Imperial College of Science, Technology and Medicine, UK 2: Department of Mathematics, Imperial College of Science, Technology and Medicine, UK,Institute for Mathematical Sciences, Imperial College of Science, Technology and Medicine, UK
Publication date: 01 March 2011
- Access Key
- Free content
- Partial Free content
- New content
- Open access content
- Partial Open access content
- Subscribed content
- Partial Subscribed content
- Free trial content