1. Introduction

Nowadays multispectral imaging techniques have been intensively studied for the applications of digital recording, archiving, and display. One of the most important tasks of multispectral imaging is to accurately recover the spectral reflectances of the object surfaces from system responses [1–7]. In the literature, three spectral characterization techniques, i.e., wiener estimation [2, 3], pseudo-inverse [1], and finite-dimensional modeling [4–7] have been widely adopted for reflectance estimation. In the wiener estimation and finite-dimensional modeling, the spectral sensitivity or responsivity needs to be instrumentally measured or mathematically recovered. The pseudo-inverse technique computes the transform between system responses and reflectance without any a priori knowledge about the imaging system.

To meet the rigorous requirement in industrial applications such as color measurement and color quality control, researchers are still working for new spectral characterization techniques. This study tries to investigate whether the accuracy of spectral reconstruction can be improved by combining the currently available techniques. In the color constancy area, Cardei and Funt found that better illumination estimation could be obtained by combing the results of different color constancy algorithms [8]. Based on statistical analysis, Stokman and Gevers investigated the optimal strategy of color model selection and fusion to improve the discriminative power of image feature detection algorithms [9]. This study is mainly inspired by these two works.

2. Formulation of multispectral imaging process

When the response of the multispectral imaging system is proportional to the intensity of light entering the camera, the response vector v of L channels can be formulated as [2]:

where the subscript N=31 denotes the number of samples in the visible wavelength range, r denotes the spectral reflectance, M represents the spectral responsivity incorporating the spectral power distribution of lighting source, the spectral transmittances of narrowband filters, and the spectral sensitivity of digital camera. b denotes the bias response vector caused by camera dark current, and n denotes zero-mean imaging noise. It is noted that if the system does not behave linearly, the optoelectronic conversion function needs to be further considered [2, 4].

In spectral characterization of imaging system, it is feasible to mathematically recover the spectral responsivity M and bias b from training color samples, subject to the constraint of non-negativeness [10, 2]. It is noted that, as the filters used in this study are narrow-banded, the condition of smoothness is not needed.

3. Three techniques for spectral characterization of multispectral imaging

3.1 Wiener estimation

In general, the purpose of spectral characterization is to estimate the reflectance vector r̂ from the response vector u=v-b through an N×L matrix W, such that

In wiener estimation [2, 3, 11], W is calculated by using the spectral responsivity M:

where superscript T denotes transpose, K _r denotes the N×N covariance matrix of r, and K _n denotes the covariance matrix of noise. In this study, K _n=0 is assumed.

3.2 Pseudo-inverse

In the pseudo-inverse technique [1], the transform matrix W is directly solved as

where the superscript + denotes pseudo-inverse, R denotes the matrix of reflectance vector r, and U denotes the matrix of response vector u.

3.3 Finite-dimensional modeling

Due to the smooth property of spectral reflectance, r can always be represented by the linear combination of J(<N) basis functions b _j [12]:

where a_j is the coefficient of b _j. b _j can be calculated using principle component analysis of reflectance data. By combining Eq. (5) and (1), the response u can then be represented as [4–7]:

Thus coefficient a_j can be estimated by pseudo-inverse of the L×J matrix [Mb _j]. The reflectance rô can then be obtained by substituting the estimated a_j into Eq. (5). In this study, J=10 basis functions are used, as they are generally adequate for spectral reflectance construction [12, 5].

4. The proposed method

4.1 Statistical properties of different observations

Suppose a quantity is measured using K (K=3 in this study) different techniques, and x_k is the observation of the kth technique. To obtain an improved estimation of that quantity, it is feasible to combine these observations as the following:

where w_k is the weighting of the observation of the kth technique. The estimate and standard deviation of x can be expressed according to Eq. (8) and (9), respectively.

where E(x_k) is the averaged observation of the kth technique, and σ(x_k, x_l) is the covariance between observations of the kth and lth techniques.

4.2 Combination of different spectral characterization techniques

Based on the statistical properties discussed above, a new method for reflectance estimation by combining different techniques is proposed in the following.

Let r̂_k be the estimated reflectance of the kth technique, the spectral root mean square (rms) error can then be calculated as

and the combined rms error can be expressed as

Similarly, the CIE 1994 color difference [13] error is represented as

and the combined color difference error then becomes

It is noted that the spectral error δr is linear to reflectance r, while the colorimetric error δe is nonlinear to r, due to the third-root-square transform between CIEXYZ space and CIELAB space. To account for both spectral and colorimetric accuracy in reflectance reconstruction, these two error terms can be merged into a single error term δ.

Consequently, the estimate of δ can then be expressed as

where 0≤c≤1 is used to adjust the proportions of spectral and colorimetric errors, E _r and E _e denote the K×1 vectors of the average spectral and colorimetric errors, respectively, and w denotes the K×1 weighting vector. Accordingly, the standard deviation of δ can be expressed as

where ∑ _r and ∑_e denote the K×K covariance matrix of the spectral and colorimetric error, respectively. Considering that the spectral and colorimetric errors are always of different magnitudes, E _r, E _e, E _r, and ∑ _e are normalized in the calculation of weighting w.

The appropriate weighting w can be obtained by solving the following objective function:

or, alternatively,

Furthermore, the following two constraints should be imposed:

The first constraint ensures that the weightings of the K techniques sum to 1, while the second constraint forces positive contribution of each technique.

The objective functions (16) and (17), together with the constraints (18) and (19), can be solved using quadric programming. When the weighting is obtained, the reflectance of the proposed method can be calculated as

It is noted that the weightings obtained from the objective functions are suboptimal, due to the nonlinear transform between reflectance and color difference error. Nevertheless, it will be shown in the experiment section that this strategy can produce considerable color accuracy improvement, when compared with the three techniques.

5. Experimental results and discussion

In this study, a QImaging monochrome digital camera model Retiga-EXi, as well as a liquid crystal tunable filter (LCTF) made by Cambridge Research and Instrument Co., were used to acquire multispectral images. The LCTF contains a series of electronically-tunable narrowband filters, with the nominal bandwidth of 10nm Full-Width at Half-Maximum (FWHM). The color target used was GretagMacBeth ColorChecker DC (CDC), and its reflectance data were measured using a GretagMacBeth spectrophotometer 7000A in the visible spectrum of 400 to 700 nm, with sampling interval of 10 nm. Totally L=16 multispectral images of CDC were taken using the filters with center wavelengths at 400 nm, 420 nm, 440 nm, …, 700nm, under an approximate D65 lighting condition. The spatial non-uniformity of the lighting field was corrected by using a white paper [4]. In total, 198 color patches on CDC were used for reflectance estimation, excluding the duplicated most dark ones and the glossy ones. These color patches were divided into one training set and one testing set, with each set containing 99 colors. The training set contains the color patches with odd sequence numbers, while the testing set contains the color patches with even sequence numbers.

 

Fig. 1. Distribution of colorimetric errors ΔE^* ₉₄ under D65 and spectral errors obtained using the objective function (16) with respect to different c values. The green square-symbol curve represents colorimetric errors, and the blue circle-symbol curve represents spectral rms errors.

Download Full Size | PDF

The magnitudes of spectral error and colorimetric error obtained through the objective function are related to the c values, as shown in Fig 1. These two kinds of errors do not change monotonically with c. However, as a whole trend, increasing the c value will produce larger color difference error, while produce smaller spectral rms error. The magnitude of color difference error keeps in the range of 0.850 and 0.855 when c increases from 0.0 to 0.6, which indicates that the color accuracy of the proposed method is not very sensitive to c value. Therefore, we simply let c = 0.2 in the following calculation.

Table 1. Spectral and colorimetric error statistics of 6 methods when c=0.2: wiener estimation, pseudo-inverse method, finite-dimensional modeling method, simple averaging method, and the proposed methods.

View Table

For comparison, the spectral and colorimetric error statistics including mean, standard deviation, and maximum of 6 different methods (wiener estimation, pseudo-inverse, finite-dimensional modeling, simple averaging, and the proposed methods with two different objective functions) are given in Table 1. In the averaging method, w ₁=w ₂=w ₃=1/3. Nonparametric statistical test indicates that the color difference errors of the proposed methods and the averaging method are smaller than those of the other three methods at a significant level p=0.05. The performance of the averaging method is close to that of the proposed method 1 using objective function (16). The advantage of the proposed method 1 over the averaging method lies that the former can balance the colorimetric accuracy and spectral accuracy by adjusting the c value. The reconstructed spectral reflectance curves with minimum and maximum color difference errors are shown in Fig.2. The proposed method 2 using objective function (17) does not yield better performance, indicating that it may be not feasible to minimize both average color error and standard deviation. Actually, minimizing standard deviation will also decrease the average color error to a certain extent.

 

Fig. 2. Reconstructed spectral reflectances of the proposed method using objective function (16). (a) the best case, with ΔE^* ₉₄=0.11 under D65, (b) the worst case, with ΔE^* ₉₄=3.68 under D65

Download Full Size | PDF

6. Conclusion

A new reflectance reconstruction method for multispectral imaging has been proposed, by combining three different spectral characterization techniques. The weightings of the estimated reflectances of these techniques are calculated based on the minimization of spectral and colorimetric errors. In terms of color difference error, the performance of the proposed method is better than those of the three techniques, while is close to that of the simple averaging method. The proposed method is applicable in industrial applications such as textile color measurement and color quality control.