Open Access
Add to BibSonomy
Abstract
The influence of light sources on digital camera-based spectral estimation is explored. The CIE daylight and non-CIE daylight illuminants with different Correlated Color Temperature (CCT) are first tested comparatively, results indicate that CCT can be used to describe the performance of the CIE daylight illuminants for spectral estimation but not applicable to all types of light sources. To further investigate the mechanism of light effects on spectral estimation, several handmade special shape of Spectral Power Distribution (SPD) are tested, results show that the red component in visible spectrum is crucial for a higher spectral estimation accuracy. Finally, several feature metrics of SPD are proposed to evaluate the performance of the light sources in spectral estimation, results illustrate that the better the feature metrics the better the spectral estimation accuracy.
© 2021 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
The surface spectral reflectance is the unique identity of an object, and at the same time, it is also regarded as the “fingerprint” of the object color information. Spectral imaging technology that recording the spectral and spatial information of the scene at the same time is always a hot research direction in many domains, such as medical application, remote sensing, color science, industry quality control, and cultural heritage protection [1–5]. Currently, digital camera-based spectral estimation has become a valuable solution for fast and high spatial resolution spectral imaging with low-cost although its accuracy is affected by some factors [6–8]. The workflow of digital camera-based spectral estimation for spectral imaging is plotted as Fig. 1.
For digital camera-based spectral estimation, the training samples with known spectral reflectance are first captured under the illumination of the light source, and the spectral estimation matrix is calculated based on the training sample data and spectral estimation algorithm (SRA) [9]. Then the testing samples are captured under the same illumination condition and its spectral reflectance is estimated. During the workflow of digital camera-based spectral estimation, the light source is an important component that influence the spectral estimation accuracy. Different light sources will lead to different camera responses, and therefore lead to different spectral estimation matrix, which in turn lead to the different spectral estimation accuracy.
In early stages of researches on digital camera-based spectral estimation, tungsten halogen (TH) lamps are often used as lighting sources as it can provide relative smooth spectral radiance over the visible spectrum and with high color rendering index (CRI) [10]. However, the heat radiation of the TH lamp will damage objects such as art and cultural relics, and the lifespan of TH lamp is short. Except for the TH lamp, the other types of light sources such as high-pressure mercury lamps, fluorescent lamps, and even the projectors are also used for digital camera-based spectral estimation [11–14]. In addition, the fast-developing lighting technology provides a different option of the lighting sources for digital camera-based spectral estimation, the light emitting diode (LED) light sources. Different SPDs can be acquired through programming the tunable LED light sources in reasonable ranges of illuminance, which provide the opportunity to design the optimal SPDs for a specific digital camera-based spectral estimation system. Furthermore, the advantages of long lifespan, high radiance efficiency and environment friendly of LED light sources are also need to be considered in practical applications.
In general, various types of light sources have been used for digital camera-based spectral estimation, but do the different types of light sources affect the spectral estimation. The answer is yes. Many researches related to the light sources selection and optimization for digital camera-based spectral estimation has been published. Tominaga et al. explored the optimal combination using programmable light sources and a high-speed monochrome camera for multispectral imaging [15]. Shrestha et al. studied how to select three light-emitting diodes (LEDs) from nineteen LEDs to built the multispectral imaging system [16]. Safdar et al. also explored the optimal LEDs combinations for digital camera-based spectral estimation [17]. Zhang et al. investigated the optimal combination of different light sources for spectral estimation from CIE tristimulus [18]. Recently, Wang et al. proposed a light sources SPD optimization method for digital camera-based spectral estimation, they found that using an optimized SPD mixed by the tunable LEDs can improve the spectral estimation accuracy [19].
Moreover, the influence of light sources on digital camera-based spectral estimation are also investigated in some papers. Vilaseca et al. illustrated that when using the optimal filter combinations for spectral estimation in near-infrared multispectral imaging, the blackbody radiation with different CCTs hardly affects the spectral estimation accuracy [20]. Li et al. studied some LEDs and several CIE standard illuminants that with different CCTs on the spectral estimation, they found that the spectral estimation accuracy of a light source with a small CCT is better than that of a light source with a large CCT [21]. Sávoli et al. tested the impact of light sources on spectral estimation form tristimulus values using the genetic optimization method and CIE light sources, the results show that spectral estimation from tristimulus values does not depend on changes in light sources [22].
However, current research about the influence of light sources on spectral estimation is insufficient as the conclusions are not always consistent with each other, and most of the current researches are based on CCT. “CCT is a one-dimensional metric that aims to quantify the perceived visual quality of nominal white light sources” [23], it can not describe the features of SPD curves, two different SPDs can share the same CCT, therefore current researches about the influence of light sources on the spectral estimation based on CCT maybe lack of some sufficiency. Furthermore, although there are some other metrics (such as CRI, GAI, CQS, MCRI, CQI and so on) are used as supplemental metrics to quantify color quality of light sources [24–26], however, these metrics are all developed based extra color samples and they are not developed based the features of SPD itself. So, although these metrics can be well used to describe the color quality of light sources, they are less suitable for evaluating the performance of a light source for spectral estimation.
With the above in mind, in this paper, we explored the influence of light sources on digital camera-based spectral estimation in a more detailed way, the research is for a three-channel camera system, and not for all kinds of digital camera-based spectral system such as the camera with filters. We step from CCT to the features of SPD curves in exploring the influences. With the support of several experiments, four metrics that can describe the features of SPD curves in evaluating the performance of a light source for spectral estimation are proposed. Further tests on the proposed metrics show that the higher the metric score the better the spectral estimation accuracy. But there is still some issues need to be resolved in the future studies. Fundamentals of digital camera-based spectral estimation are presented in section 2, details of the experiments are introduced in section 3, results and discussion of the experiments are presented in section 4, the conclusion is derived in section 5.
2. Spectral estimation model
As is illustrated in Fig. 1, for digital camera-based spectral estimation, before estimate the spectral reflectance of target samples, a training process that to construct the spectral estimation matrix is necessary. Generally, two stages are included in the normal digital camera imaging process. In the first stage, the scene radiance reflectance of object is firstly integrated as the raw image in the imaging sensor, of which is often regarded as a linear imaging process. Then in the second stage, the raw image is post-processed to acquire the visual pleased color image, this is a non-linear imaging process and hard to mathematically modeled because of the complex and secret image signal processing algorithms. Therefore, in most researches, a linear opto-electronic transfer function in the digital camera is assumed [27], and the camera response of a sample can be formulated as Eq. (1),
where the camera response di is related to the channel i of a sample, λ is the wavelength, l(λ) is the relative spectral power distribution of the light source, si(λ) is the spectral sensitivity functions of the ith channel of the digital camera, r(λ) is the spectral reflectance of a surface point in the scene to be imaged, Φ is the spectra integrated range of the digital camera, e is the system additive imaging noise and often simulated with the Gaussian white noise in different researches [28–30]. In practice, we can formulate a discrete version of Eq. (1) as where d is the response vector of a sample, M is the overall spectral sensitivity function matrix of the digital camera including the product of the matrix form of l(λ) and si(λ), and r denotes the spectral reflectance vector of a sample to be imaged, and e is the noise vector.For digital camera-based spectral estimation, the first step is to construct the spectral estimation matrix based on the digital camera model indicated by Eq. (2). Many methods have been proposed to calculate the spectral estimation matrix, such as pseudoinverse, PCA, Wiener estimation, compressive sensing and so on [31–34]. In addition, deep learning based methods, such CNN-based, U-net-based and GAN-based methods, are also introduced into spectral estimation in recent years with the fast development of computer vision technology [35–37]. In this study, taking the commonly used pseudoinverse method as example, the spectral estimation matrix is calculated as in Eq. (3),
where Q is the spectral estimation matrix, Rtrain and Dtrain are the spectral reflectance and camera response matrix of the training dataset, and superscript + is the pseudoinverse operator. With the established spectral estimation matrix Q, the spectral reflectance of a sample can be estimated as in Eq. (4). where dtest is the camera response vector of a test sample, rtest is the corresponding recovered spectral reflectance vector. The spectral estimation model is formed by Eq. (2) to Eq. (4). The commonly used method that proposed by Connah and Hardeberg is adopted to calculate the spectral estimation matrix in this study [38], the implementation details of the method can refer to the corresponding reference.3. Experiment
3.1 Overview of experiment
With the spectral estimation model, three experiments are carried out progressively to explore the influence of light sources on digital camera-based spectral estimation. In considering the current research progress about the problem, in the first experiment, we test the influence of CIE daylight illuminants with different CCTs on spectral estimation. In the second experiment, in order to test whether CCT can effectively indicate the performance of any type of light sources in spectral estimation, the influence of light sources with same CCT but difference SPDs are explored. In the third experiment, to further investigate how the light sources with different SPDs influence the spectral estimation, we test some handmade SPDs with specific shapes using the natural color system (NCS) samples.
Before the experiments, the simulated digital camera system is firstly constructed. According the linear imaging model in Eq. (1), the camera sensitivity functions of Canon 1D Mark III and Nikon D5100 are selected from Jiang’s databases to constructed the simulated digital camera systems [39], the distributions of the camera sensitivity function are plotted in Fig. 2. One percent of Gaussian white noise with SNR equal to 40 dB is added to the simulated system as additive imaging noise.30 Three sample datasets including X-rite Color charts, Munsell color sample and a set of mineral pigment color samples are used in experiment 1 and 2. The chromaticity distribution of three sample datasets in a*-b* plane of CIELAB color space are plotted in Fig. 3. The training and testing samples are set as in Table 1, where the numbers in brackets represent the number of samples in each set.
Fig. 3. The chromaticity distribution of three sample datasets in a*-b* plane of CIELAB color space, Color Chart SG (a), Color Chart CC (b), Munsell color (c), Pigment samples(d).
Table 1. Three groups of training and testing samples in experiment 1 and experiment 2.
Based on the constructed simulated digital camera systems and sample datasets, the experiments are carried out one by one. More details about tested light sources and other settings of the three experiments are illustrated as follows.
3.2 Details of experiment
Details of experiment 1. In the first experiment, the influence of CIE daylight illuminants that with different CCTs on spectral estimation are tested. The SPDs of CIE daylight illuminants are calculated with reference to the CIE specification, where the CCTs are sample from 2300 K to 15000 K with the interval of 100 K (Kelvin). Detail method to calculate the SPD of CIE daylight illuminant for a given CCT is presented from Eq. (5) to Eq. (10) [40]. The tested CCTs covers almost the various types of lighting sources in industrial production and in daily life. The SPDs of the tested CIE daylight illuminants are plotted in Fig. 4.
Fig. 4. Relative SPDs of the tested CIE daylight illuminants from 2300 K to 15000 K with the sample interval of 100 K.
Details of experiment 2. In experiment 2, two groups of light sources that with the same CCT but different SPDs are tested. The tested CCTs are ∼2856 K and ∼6500 K, which are two common used but very different CCTs in the filed of spectral imaging. The SPDs of CCT ∼2856 K including CIE-A illuminant, a tungsten halogen light source of GE Par38 (11878) (TH2856) and a LED light source of Endura OT16-3301-WW MR16 (LED2856). The SPDs of CCT ∼6500 K including the CIE-D65 illuminant, a fluorescent light source of GE F40W/AD (FL6500) and a LED light source of Toshiba E-Core Par30s 23Deg (LED6500). The other SPDs except for the CIE illuminant are collected from the National Gallery of United Kingdom [41]. These two groups tested SPDs are plotted in Fig. 5.
Fig. 5. Relative SPDs of the two groups tested light sources in experiment 2: CCT∼2856 K (a), CCT∼6500 K (b).
Details of experiment 3. In experiment 3, in order to explore the influence of light sources on spectral estimation more specifically, five handmade rectangle light sources including red, green, blue, yellow and equal energy white (E) are tested based on the samples of NCS samples. The SPDs of five handmade light sources are plotted in Fig. 6. The camera sensitivity functions of Nikon D5100 (as plotted in Fig. 7) that measured from National Physical Laboratory (UK) is adopted in the experiment to construct the simulated digital camera system [42]. Furthermore, in this experiment, all NCS samples are used as training samples, and five hue samples of red, green, blue, yellow and gray are used as testing samples. The chromaticity distribution of all NCS samples and the selected five tested hue samples in a*-b* plane of CIELAB color space are plotted in Fig. 8.
Fig. 7. The camera sensitivity functions of Nikon D5100 that measured from National Physical Laboratory (UK).
Fig. 8. Chromaticity distribution of all NCS samples and the selected five tested hue samples in a*-b* plane of CIELAB color space: all samples (a), red hue samples (b), green hue samples (c), blue hue samples (d), yellow hue samples (e), neutral gray samples (f).
To evaluate the spectral estimation accuracy, the root-mean-square error (RMSE) is calculated between the ground-truth and reconstructed spectral reflectance. Method to calculate the RMSE is illustrated in Eq. (11).
4. Results and discussion
In this section, the results of three experiments are firstly presented and analyzed. Then, the proposed feature metrics of SPD curves that used to evaluate the performance of light sources in spectral estimation is introduced and tested. Finally, we give the overall discussion on the results.
4.1 Results of experiment 1
In this experiment, the influence of CIE daylight illuminants that with different CCTs on spectral estimation accuracy are tested. Figure 9 shows the relationship between spectral estimation error and CCTs of the CIE daylight illuminants of three tested sample datasets under two simulated digital camera systems.
Fig. 9. The relationship between spectral estimation error and CCTs of the CIE daylight illuminants of three tested datasets under two simulated digital camera systems: Color chart (a), Munsell color (b), and Pigment samples (c).
It can be seen from Fig. 9 that for each group of tested sample dataset, although there is slight difference of their spectral estimation error of RMSE under the two different simulated digital camera systems, the spectral estimation error show very similar changes with the increase of CCT, where the correlation coefficient between the spectral estimation results of two simulated digital camera systems are 0.9985, 0.9903 and 0.9874 for the three tested sample sets. Therefore, it easy to infer from Fig. 9 that although different digital camera systems affect the accuracy of the spectral estimation, however in this study, they do not affect the overall law of the influence of the CIE daylight illuminants on the accuracy of the spectral estimation.
Furthermore, it is also easy to infer from Fig. 9 that for all three groups of the tested sample datasets, with the increase of CCT the spectral estimation error of RMSE is increase rapidly, when the CCT reach about 7000 K the increasing speed is slow down, and with the continue increase of CCT the RMSE tends to be stable. Therefore, the CCTs of CIE daylight illuminants that can cause significant changes in the spectral estimation accuracy is approximately in the range of 2300 K to 7000 K, which basically covers the common CCTs of lighting sources in daily life.
From the results of experiment 1 we can see that the CCT can be used to describe the performance of CIE daylight illuminants in spectral estimation. The results are consistent with those of Li et al. [21]. where a light source with a small CCT is better than that of a large CCT in a reasonable range. However, does this conclusion is applicable to any other types of light sources not belong to CIE daylight illuminants. The experiment 2 following will give out the answer.
4.2 Results of experiment 2
From Fig. 4 we can see that the SPDs of CIE daylight illuminants shows regular changes with the increase of CCT, however the relative SPDs of artificially manufactured light sources in daily life is usually quite different from that of CIE daylight illuminants (see as in Fig. 5). Therefore, in order to test whether the conclusion about CIE daylight illuminants is applicable to any other types of light sources, the influence of light sources that with the same CCT but different SPDs on spectral estimation are tested in this section.
In terms of the two tested groups of light sources with the same CCT but different SPDs, the spectral estimation error of RMSE of three sample datasets under two different simulated digital camera systems are summarized in Table 2. For more intuitively observe and analysis the data in Table 2, these data are plotted in Fig. 10.
Fig. 10. The spectral estimation error distribution of three sample datasets under two simulated digital camera systems in terms of two groups of light sources: Color Chart (a) and (d), Munsell color (b) and (e), and Pigment samples (c) and (f).
Table 2. The spectral estimation results of three sample datasets under two simulated digital camera systems in terms of two groups of light sources.
First of all, it is easy to find from Fig. 10 that the spectral estimation error of each group sample dataset under the two digital camera systems also have very similar distributions. The correlation coefficient between the spectral estimation results of two simulated digital camera systems are 0.9973, 0.9878 and 0.9976 for the three tested sample sets, respectively. The high correlation coefficient illustrate that different simulated digital camera systems have almost no influence on the investigation. Furthermore, for all the tested sample datasets, they exhibit very similar distribution of their spectral estimation errors under a specific CCT (three tested light sources) but different results between two different CCTs, which means that different tested sample datasets also have almost no influence on the investigation in this paper.
To further analysis of the results between each tested light source of each specific CCT in Fig. 10, we can see that the spectral estimation errors of the light sources are different with each other although some of them achieved very closed results. For CCT ∼2856 K, apparent difference of spectral estimation errors between LED light source (LED2856) and the other two light sources (CIE-A and TH2856). For CCT ∼6500 K, apparent difference of spectral estimation error between fluorescent light source (FL6500) and the other two light sources (CIE-D65 and LED6500). In addition, from comparison of the spectral estimation errors between LED2856 and CIE-D65 as well as FL6500 in Table 2, we can see that the light source with small CCT (LED2856) exhibit bigger spectral estimation error than the light sources with big CCT (CIE-D65 and FL6500), which is contradictory to the conclusion in experiment 1.
Based on the analysis above, we can conclude that when facing the non-CIE daylight illuminants, it is actually not suitable to evaluate the performance of a light source for spectral estimation based on CCT. Therefore, we tested the influence of light sources on spectral estimation in a more detailed way in experiment 3.
Before the discussion on the results of experiment 3, an explanation of why experiment 3 was set in this study is presented. Figure 11 shows the spectral estimation error distributions of Color Chart CC for illuminant CIE-A and CIE-D65. Since the same spectral estimation results for Color Chart CC under two different simulated digital camera systems of Nikon D5100 and Canon 1D Mark III, therefore only the results of Nikon D5100 is presented here.
Fig. 11. The spectral estimation error distributions of Color Chart CC for illuminants CIE-A and CIE-D65 under the simulated digital camera system of Nikon D5100.
We can see from Fig. 11 that for most colors in Color Chart CC, there is no significant difference in the spectral estimation error between CIE-A and CIE-D65, but for the colors with the main reflectance component located from middle to long wavelength (such as the colors of No. 1, No. 2, No. 7, No. 9, No. 12 and No. 15), the spectral estimation error of CIE-A is apparent small than that of CIE-D65.
The groundtruth and estimated spectral reflectance of color No. 7 and No. 15 under CIE-A and CIE-D65 are plotted in Fig. 12. It is obvious that the estimated spectral reflectance of color 7 and 15 are more closed to the groundtruth for CIE-A. In review the SPD profile of CIE-A and CIE-D65 that plotted in Fig. 5, we believe that the key to investigate the influence of light sources on spectral estimation lies in the curve features of the light source SPD. Thus a more detailed investigation about the influence of light sources on spectral estimation is carried out in experiment 3.
Fig. 12. The spectral reflectance distribution of color 7 (a) and 15 (b) in Color Chart CC: groundtruth and estimated with CIE-A and CIE-D65.
4.3 Results of experiment 3
In experiment 3, five handmade light sources with special shapes and five hue samples selected from NCS samples are used to investigate the influence of light sources on spectral estimation in a more detailed way. The results of experiment 3 are summarized in Table 3, and an intuitive display of the spectral estimation errors are plotted in Fig. 13.
Fig. 13. The spectral estimation errors distribution of five tested hue samples of NCS samples under five handmade light sources.
Table 3. The spectral estimation errors of five tested hue samples of NCS under five handmade light sources.
It is easy to find from Table 3 and Fig. 13 that different light sources have different effects on the spectral estimation of different hue samples, but in general, the distribution of the spectral estimation errors shows a significant results. During the five tested light sources that with special shapes, the red light source gives the best estimation accuracy overall, where the average spectral estimation error of RMSE is 1.37(%) for all five tested hue samples of NCS samples. The equal energy light source gives the secondary best spectral estimation accuracy. The green light shows the worst result. In addition, different hue samples also achieved different spectral estimation results under different lights sources. Therefore, the SPD features of a light source is the essence influence factor for digital camera-based spectral estimation.
Furthermore, for the five tested hue samples, the spectral estimation error of the red hue sample is always the largest under five light sources, and the spectral estimation error of the neutral gray sample is always the smallest. The spectral estimation errors of other three hue samples is between red and neutral gray samples, and there is basically no significant difference in spectral estimation error between them. This is an additional finding in this experiment that the color, which is inherently determined by the spectral reflectance, of the testing samples also influence the spectral estimation accuracy. To explore the influence of light sources on spectral estimation from the aspect of SPD itself, several metrics that to describe the curve features of the SPD are proposed and tested in the following section.
4.4 Proposed feature metrics of SPD
Based on the results of above experiments and our previous experience on spectral estimation, four feature metrics of SPD curves are proposed to assist in evaluating the performance of a light source in spectral estimation. These four metrics are red light ratio (RLR) that to indicated the ratio of the red component (600nm∼700 nm) in the SPD of a light source, the smoothness index (SI) that to describe the smoothness of the SPD based on the reciprocal of curvature of SPD, the integral area (IA) that to describe the integral area between the SPD and wavelength axis, and the effective wavelength number (EWN) that to indicated the wavelength number of relative spectral power values bigger than a specified threshold, where in this paper the threshold is define as 5. Details to calculate the four feature metrics of SPD are reported for Eq. (12) to Eq. (15).
The calculation of RLR is defined as in Eq. (12),
The calculation of SI is defined as in Eq. (13),
The calculation of IA is defined as in Eq. (14),
where l(λ) is the relative SPD of a light source, IA is the integral area metric of a SPD. Generally speaking, the larger the integral area of the SPD, the higher the radiant energy in each wavelength, and vice versa.The calculation of EWN is defined as in Eq. (15),
where l(λ) is the relative SPD of a light source, sum() represent the function to count the number of wavelength that meet the conditions in (), EWN is the effective wavelength number of a SPD, th is the threshold to judge whether the wavelength is an effective one. The reason why EWN is defined is because when the lower boundary of red light in Fig. 6 is set to 0 or a very small value, the spectral estimation error will increase significantly. For example, when the lower boundary of red light in Fig. 6 is set to 0, the corresponding spectral estimation errors of five different hue samples in Table 3 are increased from 1.86, 1.45, 1.24, 1.42 and 0.88 to 6.89, 6.78, 10.56, 10.78 and 9.13, respectively.Using a light sources database including 402 different SPDs published previously [43], the proposed four feature metrics of SPD curves are tested using the constructed simulated digital camera system of Nikon D5100 and the NCS samples. And all the NCS samples are used both as training and testing samples in the test. The relationship between the four proposed metrics and spectral estimation errors of 402 tested SPDs are plotted in Fig. 14, where the spectral estimation errors are calculated by averaging the errors of all SPDs whose feature metrics is greater than a certain value of the horizontal axis.
Fig. 14. Relationship between the four proposed metrics and spectral estimation errors of tested SPDs: RLR (a), SI (b), IA (c), and EWN (d), where the spectral estimation errors of RMSE are calculated by averaging the errors of all SPDs whose feature metrics is greater than a certain value of the horizontal axis.
We can see from Fig. 14 that for the defined feature metrics of SPD, the higher the metrics score the higher the spectral estimation accuracy. Therefore, the defined metrics may be used to describe the performance of a light source in spectral estimation in somehow from the aspect of SPD features rather than the CCT or other color quality features. However, there are still some issues about the proposed metrics. On one hand, the similar problem of CCT is also exist for the proposed metrics in evaluating the performance of a light source for spectral estimation, to be specifically it is that different SPDs can also share the same metrics. In addition, the fluctuation of metric EWN indicated that it is not always a effective metric but can work to some extent in evaluating the performance of a light source in spectral estimation.
On the other hand, there is a little bit conflict between the functions of RLR and IA, but this is not absolute. Therefore, in the following research, we will explore how to unify these metrics as an effective one. When the proposed metrics are unified as one, the problem of multiple SPDs corresponding to the same metric and the conflict between RLR and IA can be appropriately solved to a certain extent. In addition, how to using the proposed metrics to guide the selection, design and optimization of a light source for a specific digital camera-base spectral imaging system in practice is another important aspect of our future researches.
4.5 Discussion
The optimization of key components and influence factors for digital camera-based spectral imaging or color measurement is extensively studied recently from different aspects such as the filter choose and design [44,45], the optimization of spectral estimation algorithm [8,46], as well as the selection and optimization of light sources [17,19,47]. For digital camera-based spectral estimation from single RGB image, the optimization of lighting sources is actually a promising strategy in somehow for specific applications as the tunable LEDs can generate most of the desired SPDs currently, this has been confirmed in the research of Wang et al. [19]. However, the current optimization researches especially for the optimization of light sources are performed without prior knowledge, which means that how the light sources will influence the spectral estimation is not known initially and the optimal light source was often selected from countless possible SPDs based on exhausted searches or using some searching algorithms. This will lead to the optimization result of the optimal light source is hard to explain even though there are some reasonable explanations on the optimization results in current researches, and the explanations are result-oriented and lack of the investigation about how the light sources influence the spectral estimation.
To make the optimization or selection of the light source for digital camera-based spectral estimation more evidence-based, the influence of light sources on spectral estimation is conducted in this paper. Different from the current researches on this topic of which the investigation are based on the CCT of light sources, we investigate the influence of light sources on spectral estimation from the aspect of SPD features of the light sources.
Obviously, the influence of light sources on spectral estimation accuracy for specific digital camera-based system depend on the SPD of the light sources, and four different feature metrics of SPD are defined based on the investigation. These proposed SPD features of light sources can be used as constraints for light source optimization in order to compress the solution space of light source optimization.
To compare with the existing exhaustive search methods that without the prior knowledge to guide the search of the optimal light source, the methods that with considering the proposed features of SPD may achieve higher efficiency in searching the optimal light sources, and the search process is actually oriented by prior knowledge. But we should admit that the mechanism of how the light source influence the spectral estimation is still not fully revealed currently, and how to unify these proposed feature metrics of SPD in practice is also an important question to be resolved in the future.
In addition, it should be noted that in this paper we only investigate the influence of light source on spectral estimation from the aspect of spectral accuracy and without consideration of the colorimetric accuracy. As we all know that the calculation of colorimetric values, such as the CIELab values, from the corresponding spectral reflectance is determined by the light source and the color matching function (CMF), which is a non-linear calculation process. So the experiment results may different from the current achieved. For example, the colorimetric accuracy of CIELab color difference (ΔEab) for the CIE daylight illuminants in experiment 1 is calculated and plotted in Fig. 15. Compared with the result in Fig. 9, it is obvious that a very different result is presented for the relationship between colorimetric error (ΔEab) and CCTs of the CIE daylight illuminants.
Fig. 15. The relationship between colorimetric error (ΔEab) and CCTs of the CIE daylight illuminants of three tested datasets under two simulated digital camera systems: Color chart (a), Munsell color (b), and Pigment samples (c).
Therefore, the optimal light source for digital camera-based spectral estimation is also related to the evaluation metrics of the spectral estimation accuracy. In practice, except for to use the feature metrics proposed in this paper, the inconsistency of spectral errors and colorimetric errors should also be considered in searching or designing the optimal light source. Overall, for different applications purpose, we can combine the proposed feature metrics and different evaluation metrics (spectral or colorimetric) in searching or designing the optimal light source.
5. Conclusion
The light source is an very important factor that influence the estimation accuracy of digital camera-based spectral estimation. In this paper, an investigation about the influence of light sources on spectral estimation is conducted based on several progressive independent experiments. The investigation between CCT and spectral estimation errors indicated that the CCT can be used to describe the performance of the CIE daylight illuminants in spectral estimation, but can not be used to describe the other types of light sources. The investigation between SPD of the light source and spectral estimation errors illustrated that the influence of the light source on spectral estimation depends on the SPD itself, and the red component in visible spectrum is important for higher spectral estimation accuracy. Four feature metrics of SPDs are proposed to evaluate the performance of light sources in spectral estimation. Further tests showed that the higher the metrics score the better the spectral estimation accuracy. However, some issues about the proposed metrics need to be further investigate, such how to unify these metrics as an effective one, and how to using the proposed metrics to guide the selection, design and optimization of light source in practical applications with consideration the other factors. These issues are the key points that make the proposed feature metrics of SPDs move towards practical application in the future.
Funding
Natural Science Foundation of Hubei Province (2020CFB386); Outstanding Young and Middle-aged Scientific Innovation Team of Colleges and Universities of Hubei Province (T201807); National Natural Science Foundation of China (61575174).
Acknowledgments
The authors are indebted to professor X. X. Wan for the valuable discussion on this research.
Disclosures
The authors declare that there are no conflicts of interest related to this article.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. S. Kim, J. Kim, M. Hwang, M. Kim, S. Youn, J. E. Jang, M. Je, H. L. Dong, B. Lee, and D. L. Farkas, “Smartphone-based multispectral imaging and machine-learning based analysis for discrimination between seborrheic dermatitis and psoriasis on the scalp,” Biomed. Opt. Express 10(2), 879–891 (2019). [CrossRef]
2. T. Adão, J. Hruška, L. Pádua, J. Bessa, E. Peres, R. Morais, and J. J. Sousa, “Hyperspectral imaging: A review on UAV-based sensors, data processing and applications for agriculture and forestry,” Remote Sensing 9(11), 1110 (2017). [CrossRef]
3. M. M. Amiri, S. Garcia-Nieto, S. Morilla, and M. D. Fairchild, “Spectral Reflectance Reconstruction Using Fuzzy Logic System Training: Color Science Application,” Sensors 20(17), 4726 (2020). [CrossRef]
4. V. B. Raju and E. Sazonov, “Detection of Oil-Containing Dressing on Salad Leaves Using Multispectral Imaging,” IEEE Access 8, 86196–86206 (2020). [CrossRef]
5. J. Liang, X. Wan, Q. Liu, C. Li, and J. Li, “Research on filter selection method for broadband spectral imaging system based on ancient murals,” Color Res. Appl. 41(6), 585–595 (2016). [CrossRef]
6. R. Shrestha and J. Y. Hardeberg, “Spectrogenic imaging: A novel approach to multispectral imaging in an uncontrolled environment,” Opt. Express 22(8), 9123–9133 (2014). [CrossRef]
7. X. Zhang, Q. Wang, J. Li, X. Zhou, Y. Yang, and H. Xu, “Estimating spectral reflectance from camera responses based on CIE XYZ tristimulus values under multi-illuminants,” Color Res. Appl. 42(1), 68–77 (2017). [CrossRef]
8. J. Liang, K. Xiao, M. R. Pointer, X. Wan, and C. Li, “Spectra estimation from raw camera responses based on adaptive local-weighted linear regression,” Opt. Express 27(4), 5165–5180 (2019). [CrossRef]
9. J. Liang and X. Wan, “Optimized method for spectral reflectance reconstruction from camera responses,” Opt. Express 25(23), 28273–28287 (2017). [CrossRef]
10. G. Adinolfi, R. Carmagnola, M. Cataldi, L. Marras, and V. Palleschi, “Recovery of a lost wall painting at the Etruscan Tomb of the Blue Demons in Tarquinia (Viterbo, Italy) by multispectral reflectometry and UV fluorescence imaging,” Archaeometry 61(2), 450–458 (2019). [CrossRef]
11. S. Tominaga and N. Tanaka, “Spectral image acquisition, analysis, and rendering for art paintings,” J. Electron. Imaging 17(4), 043022 (2008). [CrossRef]
12. T. Aach, J. Brauers, and S. Helling, “Multispectral image acquisition with flash light sources,” J. Imaging Sci. Technol. 53(3), 031103 (2009). [CrossRef]
13. J. Herrera-Ramírez, M. Vilaseca, and J. Pujol, “Portable multispectral imaging system based on light-emitting diodes for spectral recovery from 370 to 1630 nm,” Appl. Opt. 53(14), 3131–3141 (2014). [CrossRef]
14. R. Khan, J. van de Weijer, D. Karatzas, and D. Muselet, “Towards multispectral data acquisition with hand-held devices,” in 2013 IEEE International Conference on Image Processing, pp. 2053–2057 (IEEE, 2013).
15. S. Tominaga and T. Horiuchi, “Spectral imaging by synchronizing capture and illumination,” J. Opt. Soc. Am. A 29(9), 1764–1775 (2012). [CrossRef]
16. R. Shrestha and J. Y. Hardeberg, “Multispectral imaging using LED illumination and an RGB camera,” in Color and Imaging Conference (Society for Imaging Science and Technology), pp. 8–13 (2013).
17. M. Safdar, M. R. Luo, Y. Wang, and X. Liu, “Multispectral imaging system based on tuneable LEDs,” in Conference: Multispectral Color Science (MCS) Symposium, pp. 130–134 (2015).
18. Y. Li, C. Wang, J. Zhao, and Q. Yuan, “Efficient spectral reconstruction using a trichromatic camera via sample optimization,” The Vis. Comput. 34(12), 1773–1783 (2018). [CrossRef]
19. L. Wang, A. Sole, J. Y. Hardeberg, and X. Wan, “Optimized light source spectral power distribution for RGB camera based spectral reflectance recovery,” Opt. Express 29(16), 24695–24713 (2021). [CrossRef]
20. M. Vilaseca, J. Pujol, and M. Arjona, “Illuminant influence on the reconstruction of near-infrared spectra,” J. Imaging Sci. Technol. 48(2), 111–119 (2004).
21. C. Li, X. Wan, W. Xie, T. Li, and J. Liang, “Effects of light source on multispectral image acquisition accuracy,” Laser J. 37(12), 49–52 (2016). (in Chinese)
22. Z. Sávoli, B. Kránicz, and A. Horváth, “Spectral reconstruction in case of different illuminants,” Ann. Faculty of Eng. Hunedoara 14(1), 45 (2016).
23. D. Durmus, “Correlated color temperature: Use and limitations,” Lighting Res. Technol. https://doi.org/10.1177/14771535211034330 (2021).
24. F. Zhang, H. Xu, and H. Feng, “Toward a unified model for predicting color quality of light sources,” Appl. Opt. 56(29), 8186–8195 (2017). [CrossRef]
25. T. Esposito and K. Houser, “Models of colour quality over a wide range of spectral power distributions,” Lighting Res. Technol. 51(3), 331–352 (2019). [CrossRef]
26. Z. Huang, W. Chen, Q. Liu, Y. Wang, M. R. Pointer, Y. Liu, and J. Liang, “Towards an optimum colour preference metric for white light sources: a comprehensive investigation based on empirical data,” Opt. Express 29(5), 6302–6319 (2021). [CrossRef]
27. J. Liang, Q. Zhu, Q. Liu, and K. Xiao, “Optimal selection of representative samples for efficient digital camera-based spectra recovery”, Color Res. Appl. https://doi.org/10.1002/col.22718 (2021).
28. N. Shimano, “Recovery of spectral reflectances of objects being imaged without prior knowledge,” IEEE Trans. on Image Process. 15(7), 1848–1856 (2006). [CrossRef]
29. S. Peyvandi, S. H. Amirshahi, J. Hernandez-Andres, J. L. Nieves, and J. Romero, “Generalized inverse-approach model for spectral-signal recovery,” IEEE Trans. on Image Process. 22(2), 501–510 (2013). [CrossRef]
30. S. Li, “Superiority of optimal broadband filter sets under lower noise levels in multispectral color imaging,” Color Res. Appl. 46(4), 783–790 (2021). [CrossRef]
31. H. Shen and J. H. Xin, “Spectral characterization of a color scanner based on optimized adaptive estimation,” J. Opt. Soc. Am. A 23(7), 1566–1569 (2006). [CrossRef]
32. B. Cao, N. Liao, and N. Cheng, “Spectral reflectance reconstruction from RGB images based on weighting smaller color difference group,” Color Res. Appl. 42(3), 327–332 (2017). [CrossRef]
33. V. Heikkinen, “Spectral reflectance estimation using Gaussian processes and combination kernels,” IEEE Trans. on Image Process. 27(7), 3358–3373 (2018). [CrossRef]
34. G. Wu, “Reflectance spectra recovery from a single RGB image by adaptive compressive sensing,” Laser Phys. Lett. 16(8), 085208 (2019). [CrossRef]
35. Y. T. Lin and G. D. Finlayson, “Physically Plausible Spectral Reconstruction,” Sensors 20(21), 6399 (2020). [CrossRef]
36. B. Arad, O. B-. Shahar, R. Timofte, L. V. Gool, L. Zhang, and M. -H. Yang, “NTIRE 2018 challenge on spectral reconstruction from RGB images,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1042–1051 (IEEE, 2018).
37. B. Arad, R. Timofte, O. B-. Shahar, Y. -T. Lin, and G. D. Finlayson, “NTIRE 2020 challenge on spectral reconstruction from an RGB image,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops (IEEE, 2020).
38. D. Connah and J. Y. Hardeberg, “Spectral recovery using polynomial models,” Proc. SPIE 5667, 65–75 (2005). [CrossRef]
39. J. Jiang, D. Liu, J. Gu, and S. Süsstrunk, “What is the space of spectral sensitivity functions for digital color cameras?” in 2013 IEEE Workshop on Applications of Computer Vision, pp. 168–179 (IEEE, 2013).
40. Commission International de l’Éclairage. Colorimetry. CIE 015:2018. Vienna: CIE, 2018.
41. The National Gallery, “Normalised Spectral Power Distribution (SPD) Curves,” (2021) http://research.ng-london.org.uk/scientific/spd/?page=spd.
42. M. M. Darrodi, G. Finlayson, T. Goodman, and M. Mackiewicz, “Reference data set for camera spectral sensitivity estimation,” J. Opt. Soc. Am. A 32(3), 381–391 (2015). [CrossRef]
43. K. W. Houser, M. We, A. David, M. R. Krames, and X. Shen, “Review of measures for light-source color rendition and considerations for a two-measure system for characterizing color rendition,” Opt. Express 21(8), 10393–10411 (2013). [CrossRef]
44. P. Xu and H. Xu, “Filter selection based on light source for multispectral imaging,” Opt. Eng. 55(7), 074102 (2016). [CrossRef]
45. Y. Zhu and G. D. Finlayson, “A Mathematical Investigation into the Design of Prefilters That Make Cameras More Colorimetric,” Sensors 20(23), 6882 (2020). [CrossRef]
46. Z. Liu, Q. Liu, G. Gao, and C. Li, “Optimized spectral reconstruction based on adaptive training set selection,” Opt. Express 25(11), 12435–12445 (2017). [CrossRef]
47. J. Zhang, Y. Meuret, X. Wang, and K. A. Smet, “Improved and robust spectral reflectance estimation,” Leukos 17(4), 359–379 (2021). [CrossRef]
