Abstract

The method of least squares has been applied to the analysis of resonance profiles described by the Beutler-Fano (BF) absorption cross section. To illustrate the method, parameters were determined for experimental data having BF profiles of (a) the asymmetric, enhanced absorption type; (b) the window type, and (c) the lorentzian absorption type. Systematic and random errors and correlation among the parameters were studied as a function of the slitwidth-to-resonance width ratio by applying the least squares fitting method to data computed by folding the BF transmission profile with a gaussian slit function. High correlation among the parameters and parameter fractional standard deviations of 10–20% for data with a standard deviation of 0.01 are an incentive to keep the slitwidth-to-resonance width ratio less than two for window type resonances. In addition it was found that the parameters describing a resonance could be determined with the greatest precision in a given time when the slitwidth-to-resonance width ratio was approximately 1. It was found that systematic errors greater than about 1% resulted if the number of mesh points used to approximate the convolution integral in the fitting procedure was less than at least two per resonance width. Also, for resonances whose width was equal to the slitwidth, systematic errors in the parameter values of about 10% resulted if the width of slit function used in the fitting procedure differed from the actual slit function by 10%.

© 1969 Optical Society of America

Nonlinear least squares analysis of atmospheric absorption spectra

E. Niple
Appl. Opt. 19(20) 3481-3490 (1980)

Numerical analysis and estimation of the statistical error of differential optical absorption spectroscopy measurements with least-squares methods

Jochen Stutz and Ulrich Platt
Appl. Opt. 35(30) 6041-6053 (1996)

Random errors in interferometry with the least-squares method

Qi Wang
Appl. Opt. 50(3) 246-252 (2011)

Accuracy of maximum likelihood and least-squares estimates in the lidar slope method with noisy data

Wynn L. Eberhard
Appl. Opt. 56(10) 2667-2685 (2017)

Image Restoration by the Method of Least Squares

Carl W. Helstrom
J. Opt. Soc. Am. 57(3) 297-303 (1967)