Software Review of Distribution Fitting Programs: Crystal Ball and BestFit Add-In to @RISK
Abstract:Simulation software programs continue to evolve and to meet the needs of risk analysts. In the past several years, two spreadsheet add-in programs added the capability of fitting distributions to data to their tool kits using classical statistical ( i.e., non-Bayesian) methods. Crystal Ball version 4.0 now contains this capability in its standard program (and in Crystal Ball Pro version 4.0), while the BestFit software program is a component of the @RISK Decision Tools Suite that can also be purchased as a stand-alone program. Both programs will automatically fit distributions using maximum likelihood estimators to continuous data and provide goodness-of-fit statistics based on chi-squared, Kolmogorov-Smirnov, and Anderson-Darling tests. BestFit will also fit discrete distributions, and for all distributions it offers the option of optimizing the fit based on the goodness-of-fit parameters. Analysts should be wary of placing too much emphasis on the goodness-of-fit statistics given their limitations, and the fact that only some of the statistics are appropriately corrected to account for the fact that the distribution parameters are also fit using the data. These programs dramatically simplify efforts to use maximum likelihood estimation to fit distributions. However, the fact that a program is used to fit distributions should not be viewed as validation that the data have been fitted and interpreted correctly. Both programs rely heavily on the analyst's judgment and will allow analysts to fit inappropriate distributions. Currently, both programs could be improved by adding the ability to perform extensive basic exploratory data analysis and to give regression diagnostics that are needed to satisfy critical analysts or reviewers. Given that Bayesian methods are central to risk analysis, adding the capability of fitting distributions by combining data with prior information would greatly increase the utility of these programs.
Document Type: Research Article
Affiliations: Harvard Center for Risk Analysis, Boston, MA
Publication date: June 1, 1999