The method of (not so) ordinary least squares: what can go wrong and how to fix them
Abstract:Most of us came to know about the method of least squares while trying to fit a curve through a set of data points. The parameters of the curve are obtained by solving a set of equations (called the normal equations). Although widely used, this approach is not foolproof and, in some cases, it can even give results that are plain wrong! This happens due to some subtleties that are often overlooked by the user. In this paper, we demonstrate, by means of a simple numerical example, what can go wrong and how to fix them. Using only elementary matrix algebra, we introduce (and show the importance of) singular value decomposition, discrete Picard condition, Tikhonov regularisation, the L-curve and the L-curve criterion in addressing the subtle points of this method so that stable and reliable results are obtained in the end.
Document Type: Research Article
Affiliations: Quantum Optics and Quantum Information Group, Theoretical Physics Division, Physical Research Laboratory, India
Publication date: 2011-05-01