Abstract

Preprocessing of near-infrared spectra to remove unwanted, i.e., non-related spectral variation and selection of informative wavelengths is considered to be a crucial step prior to the construction of a quantitative calibration model. The standard methodology when comparing various preprocessing techniques and selecting different wavelengths is to compare prediction statistics computed with an independent set of data not used to make the actual calibration model. When the errors of reference value are large, no such values are available at all, or only a limited number of samples are available, other methods exist to evaluate the preprocessing method and wavelength selection. In this work we present a new indicator (SE) that only requires blank sample spectra, i.e., spectra of samples that are mixtures of the interfering constituents (everything except the analyte), a pure analyte spectrum, or alternatively, a sample spectrum where the analyte is present. The indicator is based on computing the net analyte signal of the analyte and the total error, i.e., instrumental noise and bias. By comparing the indicator values when different preprocessing techniques and wavelength selections are applied to the spectra, the optimal preprocessing technique and the optimal wavelength selection can be determined without knowledge of reference values, i.e., it minimizes the non-related spectral variation. The SE indicator is compared to two other indicators that also use net analyte signal computations. To demonstrate the feasibility of the SE indicator, two near-infrared spectral data sets from the pharmaceutical industry were used, i.e., diffuse reflectance spectra of powder samples and transmission spectra of tablets. Especially in pharmaceutical spectroscopic applications, it is expected beforehand that the non-related spectral variation is rather large and it is important to remove it. The indicator gave excellent results with respect to wavelength selection and optimal preprocessing. The SE indicator performs better than the two other indicators, and it is also applicable to other situations where the Beer–Lambert law is valid.

Rapid detection of talc content in flour based on near-infrared spectroscopy combined with feature wavelength selection

Changhao Bao, Changhao Zeng, Jinming Liu, and Dongjie Zhang
Appl. Opt. 61(19) 5790-5798 (2022)

Impact of preprocessing methods on the Raman spectra of brain tissue

Joel Wahl, Elisabeth Klint, Martin Hallbeck, Jan Hillman, Karin Wårdell, and Kerstin Ramser
Biomed. Opt. Express 13(12) 6763-6777 (2022)

Miniature wavelength-selectable Raman laser: new insights for optimizing performance

Xiaoli Li, Helen M. Pask, Andrew J. Lee, Yujing Huo, James A. Piper, and David J. Spence
Opt. Express 19(25) 25623-25631 (2011)

Infrared Mueller matrix acquisition and preprocessing system

Arthur H. Carrieri, David J. Owens, and Jonathan C. Schultz
Appl. Opt. 47(27) 5019-5027 (2008)

Optimal preprocessing of pulses for dispersion management

Peter Y. P. Chen, Boris A. Malomed, and Pak L. Chu
J. Opt. Soc. Am. B 23(7) 1257-1267 (2006)