Open Access
Add to BibSonomy
Abstract
Determination of the surface temperature of different materials based on thermographic imaging is a difficult task as the thermal emission spectrum is both temperature and emissivity dependent. Without prior knowledge of the emissivity of the object under investigation, it makes up a temperature-emissivity underdetermined system. This work demonstrates the possibility of recognizing specific materials from hyperspectral thermal images (HSTI) in the wavelength range from 8–14 µm. The hyperspectral images were acquired using a microbolometer sensor array in combination with a scanning 1st order Fabry-Pérot interferometer acting as a bandpass filter. A logistic regression model was used to successfully differentiate between polyimide tape, sapphire, borosilicate glass, fused silica, and alumina ceramic at temperatures as low as 34.0 ± 0.05 °C. Each material was recognized with true positive rates above 94% calculated from individual pixel spectra. The surface temperature of the samples was subsequently predicted using pre-fitted partial least squares (PLS) models, which predicted all surface temperature values with a common root mean square error (RMSE) of 1.10 °C and thereby outperforming conventional thermography. This approach paves the way for a practical solution to the underdetermined temperature-emissivity system.
© 2022 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Hyperspectral imaging is an emerging field within industry and has a variety of applications. Recently, hyperspectral imaging within the visible spectrum has been used widely within the food industry for analysis and quality assurance of meat, fruit and vegetables including pathogen detection [1–3]. Similarly, the near infrared (NIR) region has been used for sorting fruit and vegetables [4,5] where specifically the detection of aflatoxin is highly valuable [6,7]. Another area with huge societal benefit is the development of robust sorting of plastic polymers, where NIR hyperspectral imaging has been shown to play a key role [8,9]. While the wide application of hyperspectral thermal imaging within industry still is underway, several other applications have been shown. These include SO2 plume detection and monitoring from active volcanos [10], mineral identification [11–13], detection of explosive residues [14] and landscape scanning [13,15,16]. Common for these studies is the use of the Telops Hyper-Cam LW [17], which is based on Fourier transform spectroscopy and a Michelson interferometer. The imaging system has a high resolution of 0.25 cm-1 and utilizes a cryogenically cooled Mercury Cadmium Telluride (MCT) detector sensitive in the range from 7.7-11.8 µm. Additionally, thermal emission spectroscopy has been applied in space throughout several missions. These include the HyTES and HyspIRI, which both are satellite spectrometers orbiting and performing landscape scanning of the Earth [18,19]. Additionally, custom-made thermal emission spectrometers were equipped on the Mars Exploration Rovers, “Spirit” and “Opportunity”, to detect minerals and rock types [20–22].
In this study a hyperspectral thermal camera based on a microbolometer sensor and a Fabry Pérot interferometer (FPI) is used to record HSTI data of a selection of materials known to have vibrational absorption bands in the thermal range. The combination of an FPI with a microbolometer sensor to form a hyperspectral thermal imaging system allow for HSTI acquisition without a cryogenically cooled sensor in contrast to the available imaging systems at present day. This increases the applicability and availability of hyperspectral thermal imaging due to a significant lower cost of production of both the interferometer and the sensor. The FPI based hyperspectral imager has previously been demonstrated to be capable of classifying samples at temperatures of 100 °C [23]. In this study both sample classification and surface temperature predictions are presented covering from 34 °C to 100 °C. The images are analysed and statistical models are used to show that specific material spectra are recognizable across a wide temperature span. Following material recognition, a material specific partial least squares (PLS) regression model is used to predict the surface temperature with a RMSE of 1.10 °C. This methodology shows that it is possible to measure the surface temperature accurately of materials with different emissivities within the same image frame. Effectively, this gives a solution to the underdetermined emissivity problem, which previously has been attempted to solve by iterative algorithms, where some claim to reach temperature predictions within two standard deviations of ±1.5 °C [24–26].
2. Method
The hyperspectral thermal imaging system used in this study is based on a QTechnology QT5022 camera fitted with a Lynred Pico 1024 Gen2 1024 ${\times} $ 768 pixel bolometer sensor, which has a spectral range between 8 µm and 14 µm. This region is labelled the long wave infrared region (LWIR) and is often used for imaging due to its low atmospheric absorption on Earth [27]. An Ophir 35 mm, germanium objective lens is used to focus the light onto the sensor and a scanning FPI is placed in front of the lens acting as a variable band-pass filter (Fig. 1). All optical surfaces between the front mirror of the FPI and the sensor itself are coated with high-efficiency broadband anti-reflective coatings. By varying the distance between the mirrors, the center wavelength of the filter is shifted. This is used to capture an HSTI by recording images at regular intervals while sweeping the cavity length – one image for every ${\approx} $ 80 nm of mirror displacement.
Fig. 1. Schematic of the FPI configuration in front of the thermal camera (not to scale). Both thermal mirrors consist of a stack of Ge and ThF4 on a ZnSe substrate, with optical thicknesses as shown in the zoom-in. The mirrors are mounted in a metal flange and the mirror distance is controlled by piezo actuators. Laser- and photodiodes mounted on a PCB around the perimeter of the mirrors keep track of the mirror separation during an imaging acquisition event.
The FPI mirrors are made from a stack of germanium (Ge) and thorium tetrafluoride (ThF4) with zinc sulfide (ZnS) acting as a binding layer. The stack is deposited on a 5 mm thick, Ø 50.8 mm (2 inch) substrate of zinc selenide (ZnSe) as depicted in Fig. 1, and the mirror has an average reflectivity of 84% in the range from 8 µm to 14 µm. During operation, the mirror separation is scanned in the range from ${\approx} $ 3.2 µm to ${\approx} $ 12.9 µm. This range covers the first three orders of the FPI at a wavelength of 8 µm. The full width at half maximum (FWHM) of the FPI transmission peak has been measured at 16 different mirror separation distances using a Shimadzu 8400s FTIR spectrometer. The average FWHM of the first order is $\lambda _{FWHM}^{(1 )} = 480 \pm 83$ nm, FWHM of the second order $\lambda _{FWHM}^{(2 )} = 280 \pm 41$ nm, and FWHM of the third order is $\lambda _{FWHM}^{(3 )} = 154 \pm 17$ nm. The indicated uncertainties correspond to the measured standard deviations.
The mirrors are mounted in a steel flange and three piezoelectric actuators are used to control the mirror separation. The mirror coating itself covers Ø 43 mm of the substrate leaving room for the light of three 655 nm laser diodes, equally spaced around the brim on the outside of the mirror, to individually interfere in the cavities formed by the uncoated substrate edges. While scanning, the intensity of the specular reflected light is modulated due to the constructive and destructive interference inside the cavities. The interferogram is detected by three photodiodes and is used to ensure that the mirrors remain parallel while scanning.
The HSTIs are arranged in a cube with two spatial axes representing each pixel in an image while the third axis is spectral, containing each image recorded while scanning the mirrors. The spectrum found in each pixel represents the intensity of the transmitted light as a function of the mirror separation of the FPI. To calibrate the mirror separation axis, three HSTIs of three different band pass filters are acquired, where the filters have the following specifications: 8226/461 nm 96%, 10224/356 nm 77% and a 11322/498 nm 92% filter. The three numbers describing each filter indicate the center wavelength, the FWHM, and the maximum transmission, respectively. The measured transmission through each bandpass filter was compared to corresponding theoretically calculated spectra to calibrate the mirror separation (MS) axis of the FPI. The theoretical signal of a single band measured by the hyperspectral camera, Stheory(MS), is calculated as
Here TFPI(${\lambda _i},\; MS$) is the theoretical transmission profile of the FPI, ${T_{BP}}({{\lambda_i}} )$ is the transmission profile of the bandpass filter, and ${R_{Pico1024}}({{\lambda_i}} )$ is the relative sensitivity of the microbolometer sensor. The transmission profile of the FPI is calculated using the transfer matrix method (TMM). Further details regarding the implementation of the TMM used in this study is elaborated in the supplemental document and a description of the theory have been reported previously in the literature [28]. A Shimadzu 8400s FTIR spectrometer has been used to measure the transmission of each bandpass filter as shown in Fig. 2(B) along with the relative sensitivity of the bolometer sensor. The FPI transmission spectrum was calculated for 400 FPI mirror distances equally spaced between 2.5 µm and 13.7 µm, and the resulting expected measurement, Stheory(MS), from the hyperspectral imaging system is shown on Fig. 2(A) as a function of the mirror separation.
Fig. 2. (A) The theoretical measured signal is plotted for 400 equally spaced mirror separations in the range 2.5-13.7 µm. Each point of the graph is calculated as the sum of the product of the simulated transmission of the FPI, the measured transmission of the corresponding bandpass filter and the relative sensor sensitivity. (B) FTIR measurements of the three bandpass filters used during these experiments with center wavelengths of 8.225 nm, 10.040 nm, and 11.322 nm. The dashed black line show the relative sensitivity of the bolometer sensor. (C) Mean spectra of the measured transmission through the three bandpass filters. (D) The red line shows the calibration curve obtained using the peak positions of the transmission through the bandpass filters. The blue dots are plotted as the observed band number of the transmission peaks in Fig. 2(C) as a function of the corresponding mirror separation derived from Fig. 2(A).
The mirror separations of the FPI corresponding to the peak positions for the theoretically expected results shown in Fig. 2(A) are shown in the table below and these values are used for the wavelength versus mirror separation calibration.
Table 1. Theoretically calculated mirror separations at the maximum FPI transmission for the 1st, 2nd, and 3rd order of each bandpass filter with center wavelengths of 8.225 µm, 10.040 µm, and 11.322 µm.
Three HSTIs were acquired with each bandpass filter in front of the lens system. The mean spectrum was found for a 100 × 100 pixel wide box in the central part of the image and the measured transmission profile for each filter is shown in Fig. 2(C). Prior to calibration the mirror separation of the FPI during image acquisition is unknown and the images are labeled with band numbers in chronological order as shown in the x-axis in Fig. 2(C). Thus, to calibrate the mirror separation for the measured HSTI the measured peak maxima are found along with their corresponding peak band number. Figure 2(D) shows the band number of the measured peaks in Fig. 2(C) plotted as a function of the mirror separation of the theoretical transmission peaks as written in Tab. 1. The true mirror separation of all bands is found by fitting a 3rd order polynomial shown as the red line in Fig. 2(D). The 3rd order polynomial is used since it follows the expected displacement profile of an open loop forward scanned piezo element. The calibration of the mirror separation axis is used on all HSTIs acquired in this study.
The samples imaged for material identification consist of an aluminum block (160 ${\times} $ 160 ${\times} $ 40 mm) covered in polyimide tape to obtain a high emissivity background. Taped to the block is a piece of sapphire (Ø 30 mm), a piece of borosilicate glass (1.5 ${\times} $ 3 inch), a piece of fused silica (Ø 50 mm), as well as a piece of Al2O3 ceramic of 95% purity (30 ${\times} $ 400 mm) (Fig. 3(B)). The aluminum block is placed against a heating element, which is used to control the temperature of the entire setup. A digital thermometer is placed in a hole drilled into the aluminum block to monitor the temperature with an uncertainty of ±0.05 °C. Images are then recorded at 20 different temperatures ranging from 27.1 ± 0.05 °C to 97.0 ± 0.05 °C with increments of ∼4 °C. These 20 HSTIs are split into two different datasets where one is used for the training set and the other is used as an evaluation set. The training set was acquired at the following aluminum block temperatures: 27.1 °C, 32.0 °C, 36.5 °C, 43.3 °C, 49.9 °C, 59.0 °C, 67.2 °C, 74.8 °C, 83.7 °C, and 92.3 °C. The evaluation HSTIs are recorded at aluminum block temperatures of 30.2 °C, 34.0 °C, 40.0 °C, 46.7 °C, 54.5 °C, 63.0 °C, 70.4 °C, 79.6 °C, 88.3 °C, and 97.0 °C. An HSTI frame of the scene is shown in Fig. 3(A) recorded at a mirror separation distance of 7.9 µm and a sample temperature of 97.0 ± 0.05 °C.
Fig. 3. (A) Image of the samples mounted on an aluminum block using polyimide tape. The image is recorded at a sample temperature of 97.0 ± 0.05 °C with a mirror separation of 7.9 µm. An area of each sample material is indicated by the colored squares and their average spectra are plotted in (C). (B) The material mask which is used to fit the logistic regression model for material recognition. The dashed red boxes mark areas where double layers of polyimide tape have been used to tape the samples to the block. These regions are excluded from the material recognition data analysis. (C) Plot of the single spectrum resulting from meaning the spectral axis of the 50 × 50 pixel wide bounding boxes in (A), where each box covers a single sample. The spectra have been offset on the y-axis for visualization purposes and the black dashed line mark the mirror separation of 7.9 µm where the image in (A) has been acquired. The legend abbreviations are the following; S: sapphire, BSG: borosilicate glass, FS: fused silica, AC: alumina ceramic, and PI: polyimide tape.
3. Results
The overall goal of these experiments was to predict the surface temperature of materials of unknown and different emissivity within the same thermal image frame. Our approach is to first recognize the imaged material on the individual pixel level using the material specific thermal emission spectrum. Following a correct material recognition prediction, the PLS regression model belonging to the observed material is then used to predict the surface temperature from HSTI data sets recorded at other temperatures. All images are standardized for the material recognition, to reduce the contributions from temperature variations of the samples. This is done by centering the spectral axis, which results in a mean of 0 as well as scaling, which makes the standard deviation of each pixel spectrum unity. If s represents the raw spectrum stored in a single pixel, the standardized spectrum, $s^{\prime}$, is thus calculated as
where $\bar{s}$ is the raw spectrum mean and $\sigma $ is its standard deviation. This means that all intensity variations of the spectra are removed, and our recognition model will therefore solely find material characteristics. It should be noted that slight differences in the Planck curve will affect standardized spectra, however, these contributions are relatively small compared to the material spectral features and therefore have negligible influence on the performance of the model.Figure 3(C) shows the mean spectra of the 50 × 50 px wide bounding boxes shown in Fig. 3(A). These bounding boxes have carefully been placed such that each box only include a single material, and thus Fig. 3(C) shows the spectrum measured for each material. The legend abbreviations match the sample materials (S: sapphire, BSG: Borosilicate Glass, FS: Fused Silica, AC: Alumina Ceramic, and PI: Polyimide Tape). The dashed line in Fig. 3(C) mark the mirror separation position of 7.9 µm and thereby show the selected band, which is shown in Fig. 3(A) While the full-size image measures 1024 ${\times} $ 768 pixel, the image is cropped to a size of 800 × 768 to remove irrelevant features and reduce processing time. All images recorded during these experiments contain 140 spectral bands within the mirror separation range between 3.2 µm and 12.9 µm.
Recognizing the different materials in the image is done using the multinomial logistic regression (MLR) [29] function implemented in scikit-learn [30]. The training data set is used to fit a MLR model for each material across the temperatures imaged in this data set. The resulting fitted model is then used as a prediction model, which calculates the probability of a spectrum belonging to a particular material class from a linear combination of each of its spectral bands. The class with the highest probability determines the material assigned to a given spectrum. The algorithm used in the optimization problem is the Newton Conjugate Gradient and the class weight is balanced to ensure equal bias for all classes regardless of their abundance. To fit and validate the model, a mask defining the location of all material classes is constructed based on one of the images from the data series which is shown in Fig. 3(B).
The fitted MLR model is validated on the evaluation data set and the results at three different temperatures are presented in Fig. 4(A)-(C) with corresponding confusion matrices in Fig. 4(D)-(F) The images shown in Fig. 4(A)-(C) are reconstructions with color codes matching the predicted material by the MLR model. The x-axes of the confusion matrices shown in Fig. 4(D)-(F) represent the predictions made by the MLR model and these are compared to the true classification indicated along the y-axis. Each row is normalized to a sum of 1, meaning that the diagonal represents the true positive rate (TPR), which is the ratio between the true positives and the sum of both true positives and false negatives. Figure 4 show that the MLR model predicts the correct material in most pixels without error. The accuracy increases with increasing temperature, which is caused by the increase of emitted radiation and thereby the signal measured by the camera. Figures 4(A). and 4D. indicate that a temperature of 30.2 ± 0.05 °C is slightly too low for the MLR model to perform well, which is primarily seen in the borosilicate glass region. However, increasing the temperature to 34.0 ± 0.05 °C causes the model to perform near perfect as seen in Fig. 4(A) and Fig. 4(B). The corresponding confusion matrices in Fig. 4(D) and Fig. 4(E) show that the prediction accuracy of borosilicate glass and fused silica increases by 40% and 19% to have TPRs of 0.94 and 0.99 at 34.0 ± 0.05 °C, respectively. Note that the regions marked by dashed red lines in Fig. 3(B) are at low temperature classified as Alumina Ceramic and then at increased temperatures classified as borosilicate glass. These regions are double layers of polyimide tape, which is not part of the MLR model and thereby explain the misclassification. A single layer of polyimide tape transmits thermal radiation at distinct wavelengths in the LWIR range and therefore the material directly under the polyimide tape alters the spectrum measured in our HSTI. This has been verified during these experiments by FTIR measurements. The region marked PI in Fig. 3(B) is therefore used alone for evaluating the polyimide tape predictions of the MLR model and the white region in Fig. 3(B) is ignored during analysis.
Fig. 4. (A)-(C) Predicted classes for HSTIs acquired at temperatures of 30.2 ± 0.05 °C (A), 34.0 ± 0.05 °C (B) and 97.0 ± 0.05 °C (C) in the evaluation set using the logistic regression classification model. The color-coded materials follow that of Fig. 3(B). (D-F) Normalized confusion matrices describing the relationship between predicted sample type and the truth described by the material mask in Fig. 3(B) at temperatures of of 30.2 ± 0.05 °C (D), 34.0 ± 0.05 °C (E) and 97.0 ± 0.05 °C (F). Each row is normalized to a sum of 1, describing the percentage of each predicted label to the true class label. S, BSG, FS, AC, and PI are abbreviations for the five materials: sapphire, borosilicate glass, fused silica, alumina ceramic and polyimide tape.
In order to predict the surface temperature of the imaged samples, a PLS regression is performed. The PLS model is fitted using the training set and the performance of the model is found using the evaluation HSTIs. The evaluation HSTIs are not standardized before fitting the PLS model, since the relative intensity between each image band functions as the primary predictor. The PLS model is typically used in data sets where many predictors are mutually correlated. This model is therefore a reasonable choice in our application, since our data structure contains 140 variables of which many correlates. This is due to the FWHM of the transmitted light through the FPI at a given wavelength being larger than the separation between two subsequent image frames in the HSTI and therefore, a significant overlap between images is present and two subsequent images are therefore mutually correlated.
Following a successful fit of the PLS model to the testing set, the model is applied to predict the surface temperature of the evaluation set. The results are presented in Fig. 5 along with the root mean square error (RMSE), which summarizes the accuracy of the temperature predictions. The graphs in Fig. 5 show the sample number, which in this case are individual pixels plotted versus the temperature of that pixel. All data values are plotted for each material and the sample numbers have been sorted by ascending temperature. The solid orange line indicates the true temperature and the solid green line indicate the predicted temperature based on the PLS model. Since our samples have differing thicknesses and thermal conductivities, it is highly likely that the sample surface temperatures are different from the temperature measured inside the aluminum block. Therefore, our system was calibrated using a temperature model for the polyimide tape where the surface temperature is assumed to equal that measured by the thermometer inside the aluminum block. This model was used on the polyimide tape taped directly to the sample surfaces to find the temperature of the polyimide tape in these areas. The temperatures measured for the PI in direct contact with the samples can then be used as an estimate of the sample surface temperature. The temperature model was based on a single image frame from the HSTI in the training and evaluation data set, and was selected at a mirror separation corresponding to ∼12 µm light where polyimide tape has a transmission of 0%. The results of the PLS predictions are shown in Fig. 5 and the RMSE of the predictions have been summarized and included in the title of each subfigure. It is seen that all materials are predicted within an acceptable RMSE of less than 1.4 °C.
Fig. 5. (A)-(E) Surface temperature predictions based on PLS models fitted on all the HSTIs in the training set. The PLS models have been fitted to the spectra of each individual material marked by the mask Fig. 2(B). Each individual figure shows the pixelwise surface temperature predictions of every material present in the image. The predictions have been sorted in order to match a stepwise increase in temperature. The solid orange line marked ${y}$ indicates the measured temperature by the thermometer, which is considered the true temperature, while the solid green line marked $\hat{y}$ indicates the predicted temperatures. The root mean squared error (RMSE) of the entire set is indicated in every plot title. (F) The PLS number of components are plotted versus the RMSE of the evaluation set. The minimum of the common RMSE curve was used to determine the number of components for the chosen PLS model.
4. Discussion
At the lowest temperature of 30.2 ± 0.05 °C, the material predictions are not ideal, with the Sapphire and alumina ceramic recognized with TPRs of 100%, the fused silica TPR of 81% and the borosilicate glass TPR of 54% as seen in Fig. 4(A). Both silica types are falsely predicted as being alumina ceramic. While the predictions are not perfect the model performs significantly better than a random guess which would result in a TPR of 20% since a total of five classes is present. The poor model performance at low sample temperatures can mainly be ascribed to a low radiance. The signal contribution of the measured samples at room temperature (23 °C) is comparable to the surroundings, and thus a relative radiance ratio of 1 is expected. The relative radiance ratio increases as the sample temperatures increases and the theoretical ratio is found by integrating the Planck distribution in the range 8-14 µm. The resulting ratio at 30.2 ± 0.05 °C is 1.12 and at 34.0 ± 0.05 °C the ratio is 1.19. Additionally, it has previously been shown that the exact mirror separation of the HSTI bands varies slightly between different data sets which has a negative effect on the classification of samples [23]. The small deviations in mirror separation between image grabbing points will effectively include extra noise in the data series, which again has a larger influence at low relative radiance ratios. This effect, combined with slightly concave mirrors, due to thin film stresses, has a strong impact on the performance of the model. At 34.0 ± 0.05 °C, all materials are correctly identified with the lowest TPR being 94% for borosilicate glass (Fig. 4(B) and Fig. 4(E)). From a substrate temperature of 34.0 ± 0.05 °C and above, the lowest TPR is 94% and at 46.7 ± 0.05 °C and above, all TPR is 100% which can be seen in Fig. S4 and S5 in the supplemental document. Thus, as the aluminum block temperature is increased from 30.2 ± 0.05 °C to 34.0 ± 0.05 °C the prediction accuracy of the borosilicate glass and fused silica samples are improved by 40%, and 19%, respectively.
At 97.0 ± 0.05 °C shown in Fig. 4(C), the model falsely predicts polyimide tape as fused silica. Since these regions do not overlap with the area marked PI in Fig. 3(B), the prediction mistakes do not show up in the presented confusion matrices in Fig. 4(D)-(F). These prediction mistakes arise from a spectral bending effect in the camera caused by the angle of incidence of the light entering the FPI. The effect is elaborated in the Supplement 1 section 3 and can be corrected by collimating the thermal light entering the FPI [31] which, however, would reduce the field of view.
The accuracy of standard thermal imaging predictions has been made measured to compare to the PLS predictions and to show the importance of the emissivity settings. Eight conventional thermal images were grabbed of the experimental setup and the polyimide tape-region was used for a linear fit of the intensity versus temperature of the aluminum block. The surface temperature for the remaining samples were then corrected in a similar manner as described earlier. A summary of the data set is presented in section 4 in the Supplement 1. Here, a table summarizes the prediction error of the surface temperature using conventional thermography. At a PI temperature of 73.2 ± 2.3 °C the sapphire would at an equivalent emitted intensity have a temperature of 88.2 ± 2.3 °C resulting in an error of 15 ± 3.3 °C. Equivalent errors are found for alumina ceramic, fused silica, and borosilicate glass with values of 1.9 ± 3.3 °C, 9.4 ± 3.3 °C, and 6.0 ± 3.3 °C respectively. Thus, comparing to the RMSE of the predictions shown in Fig. 5 the PLS models outperforms conventional thermography significantly for materials with large emissivity differences. The common RMSE, $RMS{E_{common}}$, for all samples has been calculated as
5. Conclusion
In summary, hyperspectral thermal images have been recorded of samples of polyimide tape, sapphire, borosilicate glass, fused silica, and alumina ceramic. 20 HSTIs have been recorded at different temperatures ranging from 27.1 ± 0.05 °C to 97.0 ± 0.05 °C. Half of the data set has been used to fit a logistic regression model to recognize the material characteristic spectra of the different samples. This model was used to predict the materials present on the other half of the HSTIs resulting in TPR values above 94% for all samples at temperatures of 34.0 ± 0.05 °C and above. This shows that the emission spectra of materials are measurable by a Fabry-Pérot-based hyperspectral camera and that the spectra contain significant information at temperatures as low as 34.0 ± 0.05 °C. The hyperspectral imaging system was used to predict the surface temperature of the samples present in the experimental setup based on sample specific PLS models. The models predicted the surface temperature with a common RMSE of 1.10 °C.
While the room temperature measurements showed weak predictions, we propose several improvements in order to achieve FPI-based hyperspectral images at room temperature. These include stress compensated FPI thermal mirrors in order to reduce spectral broadening due to mirror curvature and improved control software for the mirror scanning in order to get absolute mirror separations during the imaging sequence. This would allow us to align the spectral axis in the post-acquisition phase, and thereby eliminate the need for calibration. Another improvement includes a mathematical correction of the spectral bending due to the angle of the incoming thermal light which would increase the homogeneity of the measured thermal emission spectra across the entire sensor array. Such mathematical correction is beyond the scope of this work.
Lastly, the addition of overlapping RGB images would make it possible to conduct even more advanced data analysis, which would allow for classification of materials of equal thermal spectra but differing color. Additionally, the boundaries of materials can be found using edge detection from the RGB camera, which could reduce the misclassification, by binning spectra within the same boundaries.
Funding
The Innovation Fund Denmark (7038-00218B).
Acknowledgements
We acknowledge financial support from The Innovation Fund Denmark (research grant no. 7038-00218B)
Disclosures
The authors declare that there are no conflicts of interest related to this article.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
Supplemental document
See Supplement 1 for supporting content.
References
1. G. ElMasry and D. Sun, “Meat Quality Assessment Using a Hyperspectral Imaging System,” in Hyperspectral Imaging for Food Quality Analysis and Control (Elsevier, 2010), pp. 175–240.
2. D. Lorente, N. Aleixos, J. Gómez-Sanchis, S. Cubero, O. L. García-Navarrete, and J. Blasco, “Recent Advances and Applications of Hyperspectral Imaging for Fruit and Vegetable Quality Assessment,” Food Bioprocess Technol. 5(4), 1121–1142 (2012). [CrossRef]
3. E. Bonah, X. Huang, J. H. Aheto, and R. Osae, “Application of Hyperspectral Imaging as a Nondestructive Technique for Foodborne Pathogen Detection and Characterization,” Foodborne Pathog. Dis. 16(10), 712–722 (2019). [CrossRef]
4. K. B. Walsh, M. Golic, and C. V. Greensill, “Sorting of fruit using near infrared spectroscopy: Application to a range of fruit and vegetables for soluble solids and dry matter content,” J. Near Infrared Spectrosc. 12(3), 141–148 (2004). [CrossRef]
5. P. Tatzer, M. Wolf, and T. Panner, “Industrial application for inline material sorting using hyperspectral imaging in the NIR range,” Real-Time Imaging 11(2), 99–107 (2005). [CrossRef]
6. A. P. Wacoo, D. Wendiro, P. C. Vuzi, and J. F. Hawumba, “Methods for Detection of Aflatoxins in Agricultural Food Crops,” J. Appl. Chem. 2014, 1–15 (2014). [CrossRef]
7. T. C. Pearson, D. T. Wicklow, E. B. Maghirang, F. Xie, and F. E. Dowell, “Detecting Aflatoxin In Single Corn Kernels By Transmittance And Reflectance Spectroscopy,” Trans. ASAE 44(5), 1247–1254 (2001). [CrossRef]
8. A. Vázquez-Guardado, M. Money, N. McKinney, and D. Chanda, “Multi-spectral infrared spectroscopy for robust plastic identification,” Appl. Opt. 54(24), 7396 (2015). [CrossRef]
9. D. Caballero, M. Bevilacqua, and J. M. Amigo, “Application of hyperspectral imaging and chemometrics for classifying plastics with brominated flame retardants,” J. Spectr. Imaging 8, 1–16 (2019). [CrossRef]
10. A. Gabrieli, R. Wright, J. N. Porter, P. G. Lucey, and C. Honnibal, “Applications of quantitative thermal infrared hyperspectral imaging (8–14 µm): measuring volcanic SO2 mass flux and determining plume transport velocity using a single sensor,” Bull. Volcanol. 81(8), 47 (2019). [CrossRef]
11. T. L. Myers, T. J. Johnson, N. B. Gallagher, B. E. Bernacki, T. N. Beiswenger, J. E. Szecsody, R. G. Tonkyn, A. M. Bradley, Y.-F. Su, and T. O. Danby, “Hyperspectral imaging of minerals in the longwave infrared: the use of laboratory directional-hemispherical reference measurements for field exploration data,” J. Appl. Remote Sens. 13(03), 1 (2019). [CrossRef]
12. B. Yousefi, S. Sojasi, C. Ibarra Castanedo, G. Beaudoin, F. Huot, X. P. V. Maldague, M. Chamberland, and E. Lalonde, “Mineral identification in hyperspectral imaging using Sparse-PCA,” Thermosense Therm. Infrared Appl. XXXVIII 9861(418), 986118 (2016). [CrossRef]
13. S. Boubanga-Tombet, A. Huot, I. Vitins, S. Heuberger, C. Veuve, A. Eisele, R. Hewson, E. Guyot, F. Marcotte, and M. Chamberland, “Thermal infrared hyperspectral imaging for mineralogy mapping of a mine face,” Remote Sens. 10(10), 1518 (2018). [CrossRef]
14. T. A. Blake, J. F. Kelly, N. B. Gallagher, P. L. Gassman, and T. J. Johnson, “Passive standoff detection of RDX residues on metal surfaces via infrared hyperspectral imaging,” Anal. Bioanal. Chem. 395(2), 337–348 (2009). [CrossRef]
15. M. Schlerf, G. Rock, P. Lagueux, F. Ronellenfitsch, M. Gerhards, L. Hoffmann, and T. Udelhoven, “A hyperspectral thermal infrared imaging instrument for natural resources applications,” Remote Sens. 4(12), 3995–4009 (2012). [CrossRef]
16. P. J. Riggan and J. W. Hoffman, “Field applications of a multi-spectral, thermal imaging radiometer,” IEEE Aerosp. Appl. Conf. Proc.> 3, 443–449 (1999).
17. . “Telops Hyper-Cam LW,” https://www.telops.com/wp-content/uploads/2021/06/2021-hyper-cam-brochure.pdf.
18. S. J. Hook, W. R. Johnson, and M. J. Abrams, “NASA’s Hyperspectral Thermal Emission Spectrometer (HyTES),” in Thermal Infrared Remote Sensing, Sensors, Methods, Applications (2013), pp. 93–115.
19. M. J. Abrams and S. J. Hook, “NASA’s Hyperspectral Infrared Imager (HyspIRI),” in Thermal Infrared Remote Sensing, Sensors, Methods, Applications (2013), pp. 117–130.
20. S. Silverman, R. Peralta, P. Christensen, and G. Mehall, “Miniature thermal emission spectrometer for the Mars Exploration Rover,” Acta Astronaut. 59(8-11), 990–999 (2006). [CrossRef]
21. P. R. Christensen, M. B. Wyatt, T. D. Glotch, A. D. Rogers, S. Anwar, R. E. Arvidson, J. L. Bandfield, D. L. Blaney, C. Budney, W. M. Calvin, A. Fallacaro, R. L. Fergason, N. Gorelick, T. G. Graff, V. E. Hamilton, A. G. Hayes, J. R. Johnson, A. T. Knudson, H. Y. McSween, G. L. Mehall, L. K. Mehall, J. E. Moersch, R. V. Morris, M. D. Smith, S. W. Squyres, S. W. Ruff, and M. J. Wolff, “Mineralogy at Meridiani Planum from the Mini-TES experiment on the opportunity rover,” Science 306(5702), 1733–1739 (2004). [CrossRef]
22. P. R. Christensen, S. W. Ruff, R. L. Fergason, A. T. Knudson, S. Anwar, R. E. Arvidson, J. L. Bandfield, D. L. Blaney, C. Budney, W. M. Calvin, T. D. Glotch, M. P. Golombek, N. Gorelick, T. G. Graff, V. E. Hamilton, A. Hayes, J. R. Johnson, H. Y. McSween, G. L. Mehall, L. K. Mehall, J. E. Moersch, R. V. Morris, A. D. Rogers, M. D. Smith, S. W. Squyres, M. J. Wolff, and M. B. Wyatt, “Initial results from the Mini-TES experiment in Gusev crater from the Spirit rover,” Science 305(5685), 837–842 (2004). [CrossRef]
23. A. L. Jørgensen, J. Kjelstrup-Hansen, B. Jensen, V. Petrunin, S. F. Fink, and B. Jørgensen, “Acquisition and Analysis of Hyperspectral Thermal Images for Sample Segregation,” Appl. Spectrosc. 75(3), 317–324 (2021). [CrossRef]
24. H. Haixia, Z. Bing, L. Bo, Z. Wenjuan, and L. Ru, “Temperature and emissivity separation from ASTER data based on the urban land cover classification,” 2009 Jt. Urban Remote Sens. Event (2009).
25. H. Shao, C. Liu, C. Li, J. Wang, and F. Xie, “Temperature and emissivity inversion accuracy of spectral parameter changes and noise of hyperspectral thermal infrared imaging spectrometers,” Sensors 20(7), 2109 (2020). [CrossRef]
26. A. Gillespie, S. Rokugawa, T. Matsunaga, J. Steven Cothern, S. Hook, and A. B. Kahle, “A temperature and emissivity separation algorithm for advanced spaceborne thermal emission and reflection radiometer (ASTER) images,” IEEE Trans. Geosci. Remote Sens. 36(4), 1113–1126 (1998). [CrossRef]
27. S. Lord, “Gemini,” https://www.gemini.edu/observing/telescopes-and-sites/sites#Transmission.
28. S. J. Byrnes, “Multilayer optical calculations,” arXiv ID 1603.02720 1–20 (2016).
29. J. M. Hilbe, Logistic Regression Models (Chapman and Hall/CRC, 2009).
30. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, A. Müller, J. Nothman, G. Louppe, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and É Duchesnay, “Scikit-learn: Machine Learning in Python,” J. Mach. Learn. Res. 127(9), 2825–2830 (2012).
31. W. J. Marinelli, C. M. Gittins, A. H. Gelb, and B. D. Green, “Tunable Fabry-Perot etalon-based long-wavelength infrared imaging spectroradiometer,” ” Appl. Opt. 38(12), 2630–2639 (1999). [CrossRef]
