Skip to main content

A Score Regression Approach to Assess Calibration of Continuous Probabilistic Predictions

Buy Article:

$51.00 plus tax (Refund Policy)



Calibration, the statistical consistency of forecast distributions and the observations, is a central requirement for probabilistic predictions. Calibration of continuous forecasts is typically assessed using the probability integral transform histogram. In this article, we propose significance tests based on scoring rules to assess calibration of continuous predictive distributions. For an ideal normal forecast we derive the first two moments of two commonly used scoring rules: the logarithmic and the continuous ranked probability score. This naturally leads to the construction of two unconditional tests for normal predictions. More generally, we propose a novel score regression approach, where the individual scores are regressed on suitable functions of the predictive variance. This conditional approach is applicable even for certain nonnormal predictions based on the Dawid–Sebastiani score. Two case studies illustrate that the score regression approach has typically more power in detecting miscalibrated forecasts than the other approaches considered, including a recently proposed technique based on conditional exceedance probability curves.

Keywords: Calibration; Continuous predictions; Crossvalidation; Regression; Scoring rules

Document Type: Research Article


Affiliations: 1: Biostatistics Unit, Institute for Social and Preventive Medicine, University of Zurich, Hirschengraben 84, 8001 Zurich, Switzerland 2: Université Paris Dauphine, Centre de Recherche en Mathématiques de la Décision, Place du Maréchal de Lattre de Tassigny, 75775 Paris 16, France

Publication date: December 1, 2010


Access Key

Free Content
Free content
New Content
New content
Open Access Content
Open access content
Subscribed Content
Subscribed content
Free Trial Content
Free trial content
Cookie Policy
Cookie Policy
ingentaconnect website makes use of cookies so as to keep track of data that you have filled in. I am Happy with this Find out more
Real Time Web Analytics