Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Interim connection space based on colorimetric values for spectral image compression and reproduction

Open Access Open Access

Abstract

A new interim and connection space (ICS) and its reconstruction method are proposed. The proposed ICS,tD65A, consists of six colorimetric values or two sets of tristimulus values under CIE illuminant D65 and A respectively. In addition, a new spectral decomposition based on the tD65A ICS and the Wiener Estimation matrix MW was introduced for an improved spectral reconstruction. Accompanying the tD65A ICS, m important basis vectors for the metameric black space based on the new spectral decomposition, and a mapping matrix MP,k via a polynomial model of order k, were trained so that both the spectral and colorimetric accuracies for the reconstructed reflectance can be further enhanced. The proposed ICS and its reconstruction method can ensure exact colorimetric matches under two (real rather than synthetic) illuminants D65 and A, which is an advantage compared with other ICSs. The performance of the proposed method was tested and compared with five other ICSs using the NCS dataset and three spectral images respectively, using RMSE and GFC to measure the spectral accuracy, and using CIEDE2000 colour differences to measure the colorimetric accuracy under three types of illuminants (continuous, fluorescent, and LED). Performance test results showed the proposed methods outperform other ICSs in terms of both spectral accuracy and colorimetric measures (RMSE, GFC, and CIEDE2000 colour difference). Therefore, it is expected the proposed ICS and its reconstruction method can play an important role in spectral image compression and reproduction applications.

© 2022 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

1. Introduction

Spectral images, compared with traditional trichromatic RGB images, have great advantage in high-fidelity color representation and reproduction [14]. The spectral reproduction of spectral images enables colour matching under any illuminant and observer. However, spectral images need more storage space, which results in storage, communication and transformation problems. Furthermore, spectral space is not suitable for spectral image processing, gamut boundary description and mapping in spectral image reproduction. Therefore, in practice, an interim connection space (ICS) is needed for both spectral image compression [57] and reproduction [14]. In the literature, it seems the ICS concept was first proposed by Rosen and Ohta [8] in 2003 for performing interpolation in a spectral reproduction system. They ruled out the eigenvectors provided by principal component analysis (PCA) [9] as the axes for the ICS and also pointed out the dimension for the ICS cannot be more than 6 to limit the look up table for the interpolation within a reasonable size of grids. Since then, many ICSs have been proposed and most of them have six dimensions. For example, three dimensions could be the XYZ tristimulus values under CIE (International Commission on Illumination) illuminant D65 together with CIE 1931 colour matching functions. Let the n-component column vector r be the spectral reflectance and the 3 by n matrix ${W_{D65}}$ be the weighting table [10] of illuminant D65 with CIE 1931 colour matching functions (CMFs), then the 3-component column vector t formed by the XYZ values, the weighting table ${W_{D65}}$ and the reflectance $r$ satisfies:

$$t = {W_{D65}}r$$

Furthermore, if we let

$$M = {({W_{D65}})^T}{({W_{D65}}{({W_{D65}})^T})^{ - 1}}$$
and
$${r_b} = r - Mt$$
then we have
$${W_{D65}}{r_b} = {W_{D65}}r - {W_{D65}}{({W_{D65}})^T}{({W_{D65}}{({W_{D65}})^T})^{ - 1}}t = t - t = 0$$

Hence, ${r_b}$ is normally called metameric black [11].

Almost all ICSs compress reflectance r to the tristimulus vector t and they are different in the last three dimensions. For the LabPQR ICS [1], the last three dimensions are defined based on the metameric black space. Let ${u_j}$, $j = 1,2, \cdots ,n$ be the unit orthogonal basis vectors for the metameric black space of all metameric blacks ${r_b}$ for the spectral image or independent training spectral dataset obtained by principal component analysis (PCA) [9] or singular value decomposition (SVD) [12], and let ${U_m}$ be an n by m matrix formed by the first m basis vectors, then the other three dimensions of the LabPQR ICS are defined by a 3-component vector ${t_{PQR}}$ and given by:

$${t_{PQR}} = {({U_3})^T}{r_b}$$

For the XYZLMS ICS [2], the other three dimensions, denoted by a 3-component vector ${t_{LMS}}$, are defined based on three offset functions denoted here by ${l_e}(\lambda )$, ${m_e}(\lambda )$, and ${s_e}(\lambda )$. They are so defined to make up the wavelength regions not covered by the CIE CMFs. Therefore, the tristimulus value vector ${t_{LMS}}$ of the metameric black ${r_b}$ is defined by

$${t_{LMS}} = {W_{LMS}}{r_b}$$

Here, ${W_{LMS}}$ is the weighting table [10] formed by ${l_e}(\lambda )$, ${m_e}(\lambda )$, and ${s_e}(\lambda )$ and the spectral power distribution of the equal energy illuminant.

Thus, both the LabPQR and XYZLMS ICSs have a common vector t (see Eq. (1)) formed by colorimetric values or tristimulus values, which ensures that the reconstructed or reproduced reflectance ${\hat{r}_t}$ (based on $t$) defined by:

$${\hat{r}_t} = Mt$$
will be a colorimetric match under the specified illuminant since
$${W_{D65}}{\hat{r}_t} = {W_{D65}}Mt = t$$
while ${t_{PQR}}$ (see Eq. (5) for the LabPQR ICS and ${t_{LMS}}$ (see Eq. (6)) for the XYZLMS ICS are used to have a better estimation of the metameric black ${r_b}$, denoted by ${\hat{r}_b}$. Thus, it follows from the spectral decomposition, Eq. (3) that the final reconstructed reflectance $\hat{r}$ is given by
$$\hat{r} = {\hat{r}_t} + {\hat{r}_b}$$

Hence, the contribution of the compressed vector ${t_{PQR}}$ or ${t_{LMS}}$ is to have a higher spectral accuracy for the reconstructed reflectance.

To improve the spectral accuracy of the reconstructed reflectance, the LabRGB [13,14] and LabW2P [4] ICSs use 6 trigonometric functions [13,14] to form the basis vectors ${s_j}$, $j = 1,2, \cdots ,6$ for the reflectance. Thus, it is expected that, for any reflectance r we have

$$r = \sum\limits_{i = 1}^6 {{w_i}{s_i}} $$

The weights ${w_i}$ can be found using the least squares method [12]. The other three dimensions, denoted by a 3-component vector ${t_{RGB}}$, for the LabRGB ICS and ${t_{W2P}}$ for LabW2P ICS, are defined, respectively, by

$${t_{RGB}} = \left( \begin{array}{l} {w_1}\\ {w_2}\\ {w_3} \end{array} \right)$$
and
$${t_{W2P}} = \left( \begin{array}{l} {w_1}\\ {w_2}\\ P \end{array} \right)$$

For the P in Eq. (12), let ${r_{rest}}$ be defined by

$${r_{rest}} = r - \sum\limits_{i = 1}^6 {{w_i}{s_i}} $$
and let ${h_j}$, $j = 1,2, \cdots ,n$ be the unit orthogonal basis vectors for all ${r_{rest}}$ from the spectral image or independent training spectral dataset obtained by PCA (Principal Component Analysis) or SVD (Singular Value Decomposition), then P is the projection of ${r_{rest}}$ on the first basis vector ${h_1}$, i.e.,
$$P = {({r_{rest}})^T}{h_1}$$

Note that the LabRGB ICS ($t$ (see Eq. (1), and ${t_{RGB}}$ (see Eq. (11)) depends only on the given spectral reflectance. However, for the LabW2P ICS, the basis vector ${h_1}$ for determining the value P (see Eqs. (1214)) can be trained using the given spectral image or an independent training set of reflectance values.

In this paper, a new ICS and its reconstruction method for spectral compression or spectral images reproduction are proposed, which will now be described.

2. Proposed method

Motivated by the available ICSs, such as LabPQR and XYZLMS, both the colorimetric and spectral accuracies are considered for the development of the proposed new ICS.

To improve colorimetric accuracy, the proposed ICS uses two illuminants, for example CIE D65 and A. Let the 6 by $n$ matrix ${W_{D65A}}$ be formed by the weighting tables [10] of the CIE D65 and A illuminants under the CIE 1931 CMFs. The proposed ICS is simply the compressed column vector ${t_{D65A}}$ formed by 6 colorimetric values or two sets of tristimulus values for the given spectral reflectance r and is given by:

$${t_{D65A}} = {W_{D65A}}r$$

Note that for the LabPQR and XYZLMS ICSs only three colorimetric values under one illuminant (D65) (see Eq. (1)) are used, and the reconstructed reflectance ${\hat{r}_t}$ using Eq. (7) can only ensure an exact colorimetric match under one illuminant (D65). But for the proposed ICS, the 6 colorimetric values, 3 under D65 and 3 under illuminant A are used. Thus, based on the tristimulus value vector ${t_{D65A}}$, we can also estimate the reflectance (denoted by ${\hat{r}_t}$) using Eq. (7) with the matrix M (see Eq. (2)) where ${W_{D65}}$ is replaced by the weighting table ${W_{D65A}}$. It can also be verified that ${W_{D65A}}{\hat{r}_t} = {t_{D65A}}$. Hence, the reconstructed reflectance ${\hat{r}_t}$ can ensure exact colorimetric matches under both illuminant D65 and illuminant A.

To improve the spectral accuracy of the reconstructed reflectance using the proposed method, similar to the LabPQR and XYZLMS ICSs, the metameric black space is also used. We note first that the metameric black defined by Eq. (3) is based on the Cohen and Kappauf spectral decomposition from its colorimetric values [11]. The matrix M defined by Eq. (2) is also known as “matrix-R”, and depends only on the weighting table(s), and has nothing to do with the spectral property being considered. Recently, Lv et al. [15,16] found that spectral decomposition (see Eq. (3)) with the Wiener estimation “matrix-W” [17], denoted by ${M_W}$ and defined by

$${M_W} = K_r{({W_{D65A}})^T}{({W_{D65A}}K_r{({W_{D65A}})^T})^{ - 1}}$$
is better than with the matrix-R M. Here, the n by n matrix ${K_r}$ is the covariance matrix [1416] derived from the training set of reflectance from the considered spectral image or independent training set of reflectance. Thus, for the proposed ICS, an initial estimation, denoted by ${\hat{r}_t}$, to the original reflectance is defined by
$${\hat{r}_t} = {M_W}{t_{D65A}}$$

Furthermore, it follows from Eqs. (16) and (17), that the following spectral decomposition based on the colorimetric value vector ${t_{D65A}}$

$${r_b} = r - {M_W}{t_{D65A}}$$
is the metameric black for a given spectral reflectance r under both CIE illuminants D65 and A. Suppose ${R_b}$ is formed by all the metameric black ${r_b}$ (see Eq. (18)) for the reflectance either from a spectral image or an independent training reflectance set, and the SVD of the $n$ by J matrix ${R_b}$ is given by
$${R_b} = UDV$$

Here, $U$ and $V$ are n by n and J by J orthogonal matrices respectively, and D is an n by J zero matrix except that the diagonal elements or singular values ${\sigma _i} = D(i,i)$, $i = 1,2, \cdots ,n$,being non-negative and monotonically decreasing. Thus, ${\sigma _1}$ is the largest, i.e. the first column vector of $U$ is the most important basis vector, ${\sigma _2}$ is the second largest, i.e. the second column vector of U is the second most important basis vector, and so on. Once again, ${U_m}$ is the matrix formed by the first m column vectors of U, thus for any metameric black ${r_b}$ an estimated metameric black ${\hat{r}_b}$ is given by

$${\hat{r}_b} = {U_m}{q_m}$$
with
$${q_m} = {({U_m})^T}{r_b}$$

Because the proposed ${t_{D65A}}$ ICS already has 6 colorimetric values, for the given reflectance r, we cannot keep any more information from the coefficient vector ${q_m}$ from the metameric black for improving the spectral accuracy. However, we can train a mapping matrix based on a polynomial model [18] to predict the coefficient vector ${q_m}$ from the ${t_{D65A}}$ ICS. If k is the order of the polynomial, then the column vector ${f_{t,k}}$ [18] can be obtained from ${t_{D65A}}$. Hence the mapping matrix, denoted by ${M_{P,k}}$ is chosen so that

$${q_m} = {M_{P,k}}{f_{t,k}}$$

Let ${F_k}$ and ${Q_m}$ be formed from all ${f_{t,k}}$ and ${q_m}$ from the training set, and then the mapping ${M_{P,k}}$ in the least squares sense [12] is given by

$${M_{P,k}} = {Q_m}{({F_k})^T}{({F_k}{({F_k})^T})^ + }$$

Here, the “+” in the superscript means the generalized inverse [12] of the matrix ${F_k}{({F_k})^T}$. If ${F_k}{({F_k})^T}$ is nonsingular, the generalized inverse becomes the normal inverse, i.e., ${({F_k}{({F_k})^T})^ + } = {({F_k}{({F_k})^T})^{ - 1}}$.

Thus, for the decoding, reconstruction, or reproduction stage for the proposed ${t_{D65A}}$ ICS, first the estimated reflectance ${\hat{r}_t}$ can be obtained using Eq. (17). Then the vector ${f_{t,k}}$ can be computed from the ${t_{D65A}}$ ICS using the polynomial model [18], and the coefficient vector ${q_m}$ can be computed using Eq. (22). Hence, the estimated metameric black ${\hat{r}_b}$ can be obtained using Eq. (20). Finally, the reconstructed or decoding reflectance $\hat{r}$ is given by

$$\hat{r} = {\hat{r}_b} + {\hat{r}_t}$$

It is expected that the proposed ICS and its reconstruction method can generate reflectances with higher colorimetric and spectral accuracies.

For easy understanding and implementing the proposed ICS and its reconstruction, a data flow chart is shown in Fig. 1 using a matrix notation. The top part enclosed by red dashed line is the training stage, starting with the training spectral matrix denoted by ${R_{TR}}$ and weighting table ${W_{D65A}}$ under two viewing conditions, ending with Wiener Matrix ${M_W}$, basis vectors ${U_m}$, and matrix ${M_{P,k}}$ together with ${W_{D65A}}$ enclosed in the blue solid box for compression and reconstruction. The middle part enclosed by green dashed line is the compression stage. The bottom part enclosed by blue dashed line is the reconstruction stage.

 figure: Fig. 1.

Fig. 1. Data flow chart for the training (top red dashed part), compression (middle green dashed part), and reconstruction (bottom blue dashed part) for the proposed ICS.

Download Full Size | PDF

To complete this section, we note that Urban et al. [19] in 2008 used the multiple illuminant colorimetry approach for the gamut mapping to improve the spectral reproduction in printing. We also note that, in 2012, a similar ICS named as ICS-2SI was proposed by Zhang et al. [3]. For a given reflectance r, the compressed colorimetric value vector denoted by ${t_{2SI}}$, is also given by

$${t_{2SI}} = {W_{2SI}}r$$

In this instance, ${W_{2SI}}$ is the weighting table of two synthetic illuminants derived from spectral power distributions of 29 popular illuminants including CIE illuminants A, the CIE daylight series [20], and some LED illuminants, using the PCA. However, the reconstruction method is different. If R is the matrix formed by all reflectances from the spectral image, or an independent reflectance set, and ${U_6}$ is the matrix formed by the first 6 most important basis vectors for the reflectance set (R) found by PCA or SVD, then the reconstructed reflectance $\hat{r}$ for ICS-2SI is given by

$$\hat{r} = {({W_{2SI}}{U_6})^{ - 1}}{t_{2SI}}. $$

3. Experimental setup

To test the performance of the proposed ICS and its reconstruction method, and compare it with other alliable ICSs, spectral datasets, spectral and colorimetric accuracy measures are needed.

3.1 Spectral datasets

To test the performance of the proposed ICS and its reconstruction method, the NCS (Natural Color System) set of reflectances, and three spectral images (IM1, IM2, IM3) from Rochester Institute of Technology Studio for Scientific Imaging and Archiving of Cultural Heritage, as shown in Fig. 2, were used for the testing. The NCS set of reflectances was obtained from spectral measurements of the 1750 NCS color chips between 400 nm and 700 nm at 10 nm intervals. The spectral reflectances of the spectral images were all between 430 nm and 700 nm at 10 nm intervals. The image resolution was 120 by 120 pixels for IM1, and 160 by 160 pixels for IM2 and IM3. In addition to the NCS dataset and the three spectral images, an independent training dataset (the Munsell dataset) of reflectances measured from 1560 Munsell color chips was used, also measured between 400 nm and 700 nm at 10 nm intervals.

 figure: Fig. 2.

Fig. 2. Spectral images from an oil painting for performance testing

Download Full Size | PDF

3.2 Spectral accuracy measure

Let r be the original reflectance, and $\hat{r}$ be the reconstructed reflectance from the proposed, or any other, ICS. Both the root-mean-square error (RMSE) [2123] and the goodness of fit coefficient (GFC) [23,24] between r and $\hat{r}$ are used as measures of spectral accuracy. For the RMSE, the smaller it is, the better the considered method or ICS performs; for the GFC, the considered method performs better if the nearer the value is to unity.

3.3 Colorimetric accuracy measure

Let tristimulus values $XY{Z_r}$ and $XY{Z_{\hat{r}}}$ be computed from r (original) and $\hat{r}$ (rconstructed) under any of the test illuminants, together with the CIE 1931 CMFs. CIEDE2000 colour difference $\Delta E_{00}^{}$ [20] between $XY{Z_r}$ and $XY{Z_{\hat{r}}}$ is used to assess the colorimetric accuracy. Three types of illuminants are used: continuous illuminants including CIE D65, A, C, D50, D55 and D75, fluorescent illuminants including CIE F1-F12 illuminants, and LED illuminants including CIE B1, B2, B3, B4, B5, VH1, RGB1, V1 and V2 illuminants [20]. The least squares or LWL weighting table [10] is used for computing tristimulus values.

4. Performance of the proposed method

In this section the proposed ICS and its reconstruction method are tested and compared with other available ICSs including LabPQR, XYZLMS, LabRGB, LabW2P and ICS-2SI using each of the NCS test dataset, and three spectral images IM1, IM2, IM3(see Fig. 2). Recently, ICC (International Color Consortium) [25] developed a set of 6 sensors for spectral compression and reconstruction using iccMAX based color management. These “sensors” distribute evenly in the visible wavelength range as shown in Fig. 3(a) and can be considered as weighting table similar to that under two viewing conditions. Thus, we can use this set of weighting table to compress the spectral data, and then we use the same reconstruction method used in the proposed ICS. This new ICS is named as ICC-6S and is also included for comparison. For comparison reason, the ‘sensors’ (combination of color matching functions and spectral power distributions of illuminants D65 and A) for the proposed ICS is also shown in Fig. 3(b). In the literature [2,3], to test and compare the ICSs, all parameters and basis functions were trained using the Munsell dataset as an independent training set, then the performance of each ICS was tested using the test datasets. However, we propose that it is better that all parameters and basis functions should be trained using the test dataset itself since when we consider the spectral compression, we already know the spectral dataset or image. Therefore, the performance of all ICSs were tested with all parameters and related basis functions trained by both the independent Munsell dataset, and the test dataset itself respectively. In addition, it follows from Eqs. (2023) that the proposed method depends on parameters $m$: the number of basis vectors used for the metameric black space, and $k$: the order of the polynomial used for transferring ${t_{D65A}}$ to ${f_{t,k}}$ [18]. It was found that the proposed method with $m = 6$, $k = 3$ performed the best when the NCS dataset was used for testing, while when the spectral Images 1-3 were used for testing, the proposed method with $m = 6$, and $k = 2$ performed the best. Hence, in the following tests, these values were used for the NCS dataset, and the spectral images respectively.

 figure: Fig. 3.

Fig. 3. ICC sensors (a) and sensors (b) for the proposed ICS.

Download Full Size | PDF

Table 1 lists the spectral accuracy test results in terms of average (Ave), the worst (Worst) and the median (Med) indices of RMSE and GFC measures respectively, using the NCS test dataset trained by the Munsell dataset (grey background) and the NCS test dataset (white background). The worst index for the RMSE measure is the maximum values of all RMSE errors, while the worst index for the GFC measure is the minimum values of all GFCs. The bold values in this, and the following tables indicate the methods (see the first column) that perform the best compared with the other methods within the same background under the index given at the top of the column considered. There are 6 indices in the table, and if all methods are ranked (from number 1 to number 7) according to each of the indices the sums of the rankings for each method are 28, 29, 31, 26, 14, 27, and 11 respectively (in the order of the seven methods listed in the Table). The smaller the number is, the better the corresponding method performed. Overall, the proposed method performed the best, and LabW2P performed the second best when trained using the Munsell dataset. Similarly, according to the sums of rankings, the proposed method performed the best, and ICS-2SI peformed the second best when trained using the NCS test dataset.

Tables Icon

Table 1. Spectral accuracy in terms of average (Ave), worst (Worst), and median (Med) values of the RMSE and GFC measures for each of the methods considered using the NCS test dataset and trained by the Munsell reflectance dataset (grey background) and NCS test dataset (white background).

Comparing results in Table 1, it can be seen that the LabRGB method performed the same whether trained by the Munsell dataset or the test dataset itself, which is to be expected since this ICS does not need training. The proposed method performed better when trained using the test dataset.

Table 2 lists the colorimetric accuracy test results in terms of the average (Ave), the worst (Worst) and the median (Med) values of CIEDE2000 colour differences using the NCS test dataset trained by the Munsell dataset (grey background) and the NCS test dataset (white background). Again, the worst value for CIEDE2000 colour difference is the maximum value of all $\Delta E_{00}^{}$ values. Since three different types of illuminants were used, the results are reported separately for each type. Firstly, the average, worst, and median values of $\Delta E_{00}^{}$ errors under each of the illuminants in each type were computed. The results in Table 2 are the averages of the average, worst and median values among all illuminants for each type of illuminants. For the continuous illuminants, there are 6 illuminants: CIE D65, A, C, D50, D55 and D75. For the proposed method, exact matches could be ensured under illuminants D65 and A. Hence, the averages were calculated using the remaining 4 illuminants C, D50, D55 and D75. For the LabPQR, XYZLMS, LabRGB, and LabW2P ICSs, an exact match could be ensured under illuminant D65, hence the averages were taken using the remaining 5 illuminants, A, C, D50, D55 and D75. While, for the ICS-2SI and ICC-6S, the averages were taken over all the 6 illuminants. According to the sums of rankings, the proposed ICS performed the best, the ICC-6S performed the second best either trained by the Munsell or the NCS datasets.

Tables Icon

Table 2. Colorimetric accuracy in terms of average (Ave), worst (Worst), and median (Med) values of CIEDE2000 colour difference for each of the methods considered using the NCS test dataset with training by the Munsell reflectance dataset (grey background) and NCS test dataset (white background).

Tables 3 and 4 list the spectral and colorimetric comparison results using IM1 for testing trained by the Munsell dataset and IM1 itself. It can be seen from Table 3 that when all methods were trained using the Munsell dataset, overall the proposed method performed the best, ICC-6S and LabPQR performed the second best according to the sums of rankings. When trained using the test IM1, the proposed method performed the best, the ICC-6S performed the second best according to the sums of rankings. Table 4 shows the proposed method performed the best, the ICC-6S performed the second best when trained using the Munsell and test IM1 datasets.

Tables Icon

Table 3. Idem to Table 1, except the test dataset being spectral image IM1

Tables Icon

Table 4. Idem to Table 2, except the test dataset being spectral image IM1

Tables 5 and 6 list the spectral and colorimetric comparison results using IM2 for testing trained by the Munsell dataset and IM2 itself. It can be seen from Table 5 that when all methods were trained using the Munsell dataset, the LabRGB performed the best, and the proposed method performed the second best, according to the sums of rankings. Similarly, when all methods were trained using the test IM2, the proposed method performed the best, and the ICC-6S performed the second best.

Tables Icon

Table 5. Idem to Table 1, except the test dataset being spectral image IM2

Tables Icon

Table 6. Idem to Table 2, except the test dataset being spectral image IM2

Table 6 shows that when all methods were trained using the Munsell dataset, the proposed method performed the best, the LabW2P performed the second best according to the sums of rankings. Similarly, when all methods were trained using the test IM2, the proposed method performed the best, and the ICC-6S performed the second best.

Tables 7 and 8 list the spectral and colorimetric comparison results using IM3 for testing trained by the Munsell dataset and the IM3 itself. It can be seen from Table 7 that when all methods were trained using the Munsell dataset, the LabW2P performed the best overall and the proposed method performed the second best according to the sums of rankings. Similarly, when all methods were trained using the IM3 itself, the proposed method performed the best and the ICC-6S performed the second best. Table 8 clearly shows the proposed method performed the best and the ICC-6S performed the second best according to the sums of rankings either trained using the Munsell dataset or the IM3 itself.

Tables Icon

Table 7. Idem to Table 1, except the test dataset being spectral image IM3

Tables Icon

Table 8. Idem to Table 2, except the test dataset being spectral image IM3

Table 9 summaries the performance of the proposed method tested using the NCS, and spectral images IM1-IM3 and trained using both the Munsell and test datasets respectively. The proposed method performed either the best or the second best in terms of the spectral accuracy measures, and the best in terms of the colorimetric accuracy measure when trained using the independent Munsell dataset. When trained using the test dataset itself, the proposed method performed the best in terms of both the spectral and colorimetric accuracies. In the brackets of the table, the best or the second best ICSs are listed. It can be seen the ICC-6S performed the second best in many cases, which could be due to both the choice of sensors and the reconstruction method used. The proposed ICS and the ICC-6S are different only in terms of “sensors” (see Fig. 3). Therefore, ensuring the exact colorimetric matches under two viewing conditions may be more important than evenly distributed sensors in the visible wavelength range.

Tables Icon

Table 9. Performance summary for the proposed method tested using the NCS dataset, IM1,IM2,IM3 and trained using both the Munsell dataset and test dataset, respectively.

Before this section is ended, several notes are needed.

Firstly, we note that for all ICSs there is no restriction about the reconstructed reflectance must be between 0 and 1. Hence, the reconstructed reflectance may have components beyond 0 and 1. Let N be the number of reflectances in a test dataset. Thus, there are $n \times N$ components in test dataset. Reflectance boundary checking summary in terms of percentage are listed in Table 10. It can be seen that the physically non-plausible reconstructed reflectances depend on the ICS and the dataset. The worst case is about 8% beyond 0 and 1 range when the LabRGB was used and tested using the IM1 dataset. There is no physically non-plausible reconstructed reflectance for the proposed ICS when trained using each of the test dataset.

Tables Icon

Table 10. Reconstructed reflectance boundary checking summary for all ICSs tested using the NCS dataset, IM1, IM2, IM3 and trained using both the Munsell dataset (grey background) and test dataset (white background) respectively.

Secondly, we note that the reflectance reconstruction approach for the proposed ICS can be applied to other ICSs for further improvements. Research towards this direction is underway.

Thirdly, we note that above tests show the performances of all ICSs except LabRGB depend on the choice of training dataset or depend on the similarity between the training and test datasets. In fact, the similarity can be measured in terms of principle angels [26] between the subspaces spanned by most relevant eigenvectors of each dataset. Let d be the dimension of the considered subspaces, and ${\theta _j}$ be the principal angles, $j = 1,2, \cdots ,d$, and let

$$S(TE,TR) = \frac{1}{d}\sum\limits_{j = 1}^d {\cos ({\theta _j})} $$

Here, $TE$, and $TR$ represent the test and training datasets respectively, and $S(TE,TR)$ is between 0 and 1. When $S(TE,TR)$ is close to 1, the similarity between the two subspaces is higher. For the proposed ICS, $d = 6$. If $TE$ and $TR$ are the same, then $S(TE,TR) = 1$. While, when $TR$ is the Munsell dataset, $S(TE,TR)$ equals to 0.98 for $TE$ being NCS, 0.95 for $TE$ being IM1, 0.98 for $TE$ being IM2, and 0.82 for $TE$ being IM3 respectively. It seems there is no way to have a universal training dataset $TR$ so that $S(TE,TR) = 1$ for any test dataset $TE$. Hence, it is recommended that the spectral image (or test dataset) is chosen as training dataset for the proposed and other ICSs.

Fourthly, we note Nakaya and Ohta [13] also used the standard deviation of spectral reflectance estimation versus wavelength as a measure for the spectral accuracy, which can also be used for investigating how the choice of sensors is impacted for each wavelength. Figure 4 shows the standard deviation versus wavelength for each ICSs under each of the test datasets. Solid and dotted curves indicate the associated methods were trained using the Munsell dataset and test dataset respectively. It can be seen overall that the ICC-6S and proposed ICS are better than others when trained using both the Munsell dataset (solid curves) and the test datasets (dotted curves) for wavelength less than 650 nm. Surprisingly, for wavelength greater than 650 nm, the ICC-6S performed worse than others especially when trained using the Munsell dataset. It can be seen that both the proposed ICS and the ICC-6S give larger errors in longer wavelength range (greater than 660 nm). Fortunately, this range has little effect on visual system.

 figure: Fig. 4.

Fig. 4. Standard deviation versus wavelength for each ICSs under each of the test datasets. Same colored solid and dotted curves indicate the associated method was trained using the Munsell dataset and test dataset respectively.

Download Full Size | PDF

Fifthly, we note that the proposed ICS performs better due to both the compression space and reconstruction method. For the compression, the ICS $t$ consists of two set of tristimulus values. Thus, the basic reflectance ${\hat{r}_t}$ (see Eqs. (17) and (24)) obtained via the Wiener matrix can ensure the exact matches under two viewing conditions, which is an advantage over other considered ICSs, and improves both the colorimetric and spectral accuracies. For the reconstruction, a matrix ${M_{P,k}}$ is trained via the polynomial model for predicting the metameric black ${\hat{r}_b} = {U_m}{M_{P,k}}{f_{t,k}}$, which improves not only the spectral but also the colorimetric accuracies for the reconstructed reflectance $\hat{r} = {\hat{r}_b} + {\hat{r}_t}$ (see Eq. (24)). However, the proposed ICS needs extra m by 84 (with m = 6 and k = 3) matrix ${M_{P,k}}$ (see Eq. (23)) and n by 6 basis vectors ${U_m}$ (see Eq. (20)) for the reflectance reconstruction compared with other ICSs. Considering current computer storage space and large amount of data for a spectral image, it is worthwhile to have a little more parameters for a better reconstruction accuracy.

Finally, we note also that a lower dimension ICS is desirable for spectral colour management [8]. Recently, Le Moan et al. [27] proposed a 5-dimension ICS for the spectral color management. The proposed ICS and LabPQR can also reduce to 5-dimension ICS by dropping the Y value for example in the second set of tristimulus values for the proposed ICS and the R value for the LabPQR. Research towards a lower dimension ICS is underway.

5. Conclusions

A new ICS and its reconstruction method were proposed. The proposed ICS ${t_{D65A}}$ (see Eq. (15)) consists of six colorimetric values or two sets of tristimulus values under CIE illuminants D65 and A. For improving the spectral accuracy of the reconstructed or reproduced reflectance, a new spectral decomposition based on the colorimetric values (${t_{D65A}}$, Eq. (18)) was considered, where the Wiener estimation matrix ${M_W}$ (Eq. (16)) rather than the “matrix-R” (Eq. (2)) was used, so that ${M_W}{t_{D65A}}$ is a better estimation to the original reflectance r. Based on the new spectral decomposition, m main basis vectors ${U_m}$, (Eq. (20)) for the metameric black space were found using SVD and a mapping matrix ${M_{P,k}}$ was also trained via a polynomial model [18] so that ${U_m}{M_{P,k}}{f_{t,k}}$ was a better estimation to the metameric black ${r_b}$ associated with the new spectral decomposition (Eq. (18)) for the original reflectance r. Here, ${f_{t,k}}$ is a column vector transformed from the ICS ${t_{D65A}}$ via the polynomial of order k. Finally, the reconstructed reflectance $\hat{r}$ from ICS ${t_{D65A}}$ is simply $\hat{r} = {M_W}{t_{D65A}} + {U_m}{M_{P,k}}{f_{t,k}}$ (Eqs. (17), (20), (22), (24)). The first term ${M_W}{t_{D65A}}$ is to ensure exact colorimetric matches under the CIE illuminants D65 and A, and the second term ${U_m}{M_{P,k}}{f_{t,k}}$ is to further improve the spectral accuracy for the reconstructed reflectance $\hat{r}$.

The proposed ICS and its reconstruction method can ensure exact colorimetric matches under two (real rather than synthetic) illuminants D65 and A, which is an advantage over other methods. Besides, XYZ can be transformed to a uniform CIELAB color space, hence the proposed ICS can be considered more uniform than other ICSs, which will benefit for the use of interpolation for spectral reproduction.

The performance of the proposed method was tested and compared with five other methods using the NCS dataset and three spectral images (Fig. 2) respectively in terms of the RMSE and GFC for the spectral accuracy measure and in terms of CIEDE2000 colour differences for the colorimetric accuracy under three (continuous, fluorescent, and LED) types of illuminants. For each measure three (average, worst and median) statistical information were used.

Firstly, it was found the best choice for the number of basis vectors or parameter m for the proposed method was 6, while the best choice for the polynomial order or parameter k for the proposed method was 3 when the NCS dataset was used for testing, and 2 when the three spectral images were used for testing.

Secondly, when all methods were trained using the independent Munsell dataset, it was found that the proposed method performed either the best or second best in terms of spectral accuracy measures (RMSE and GFC), and performed the best in terms of colorimetric accuracy measure ($\Delta {E_{00}}$).

Thirdly, when all methods were trained using each test dataset, the proposed method performed the best in terms of both the spectral accuracy measures (RMSE and GFC), and colorimetric accuracy measure ($\Delta {E_{00}}$).

Therefore, it is to be expected that the proposed ICS and its reconstruction method can play an important role in spectral image compression and reproduction applications.

Funding

National Natural Science Foundation of China (61575090, 61775169); Department of Education of Liaoning Province (LJKQZ2021127, LJKZ0291, LJKZ0310); University of Science and Technology Liaoning (2021YQ04, LKDYC202103).

Acknowledgments

The authors thank Profess Mike Pointer for reading and improving the manuscript.

Disclosures

The authors declare no conflicts of interest.

Data Availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

1. M. W. Derhak and M. R. Rosen, “Spectral colorimetry using LabPQR—an interim connection space,” J. Imaging Sci. Technol. 50(1), 53–63 (2006). [CrossRef]  

2. X. D. Zhang, Q. Wang, Y. Wang, and H. Z. Wu, “XYZLMS interim connection space for spectral image compression and reproduction,” Opt. Lett. 37(24), 5097–5099 (2012). [CrossRef]  

3. X. D. Zhang, Q. Wang, J. C. Li, P. Yang, and J. Y. Yu, “The interim connection space based on human color vision for spectral color reproduction,” J. Opt. Soc. Am. A 29(6), 1027–1034 (2012). [CrossRef]  

4. W. Liang, W. Hao, X. X. Li, Y. H. Wang, and X. H. Yang, “Multispectral Image LabW2P Codec for Improvement of Both Colorimetric and Spectral Accuracy,” Spectroscopy and Spectral Analysis 39(6), 1823–1828 (2019). [CrossRef]  

5. R. Ciprian and M. Carbucicchio, “Colorimetric–spectral clustering: a tool for multispectral image compression,” J. Opt. 13(11), 115402 (2011). [CrossRef]  

6. F. Agahian and B. Funt, “Spectral compression using subspace clustering,” Color Research & Application 41(1), 7–15 (2016). [CrossRef]  

7. G. Y. Wu, Z. Liu, E. Y. Fang, and H. Q. Yu, “Reconstruction of spectral color information using weighted principal component analysis,” Optik 126(11-12), 1249–1253 (2015). [CrossRef]  

8. M. R. Rosen and N. Ohta, “Spectral Color Processing using and Interim Connection Space,” IS&T and SID Eleventh colour and imaging conference, November 3, (Scottsdale, Arizona, USA, 2003), pp. 187–192.

9. A. M. Martinez and A. C. Kak, “PCA versus LDA,” IEEE Trans. Pattern Anal. Machine Intell. 23(2), 228–233 (2001). [CrossRef]  

10. C. J. Li, M. R. Luo, M. Melgosa, and M. R. Pointer, “Testing the Accuracy of Methods for the Computation of CIE Tristimulus Values Using Weighting Tables,” Color Res. Appl. 41(2), 125–142 (2016). [CrossRef]  

11. J. B. Cohen, “Color and color mixture: Scalar and vector fundamentals,” Color Res. Appl. 13(1), 5–39 (1988). [CrossRef]  

12. G. H. Golub and C. F. Van Loan, Matrix Computation, 3th ed. (The Johns Hopkins University Press, Baltimore and London, 1996).

13. F. Nakaya and N. Ohta, “Spectral encoding/decoding using LabRGB,” in Fifteenth IS&T Colour Imaging Conference (Society for Imaging Science and Technology, 2007), pp. 190–194.

14. F. Nakaya and N. Ohta, “Applying LabRGB to real multi-spectral images,” in Sixteenth IS&T Colour Imaging Conference (Society for Imaging Science and Technology, 2008), pp. 289–294.

15. C. Lv, C. J. Li, Y. Xu, H. Y. Sun, C. Gao, and X. H. Zhang, “A new Spectral Compression Method Based on the Minimization of the Colour difference and Root Mean Square Error,” Journal of Imaging Science and Technology, Accepted in 2022.

16. C. Lv, C. J. Li, H. Y. Sun, and C. Gao, “A new interim connection space MLabPQR for spectral image compression and reconstruction,” Spectroscopy and Spectral Analysis, Accepted in 2022.

17. F. H. Li and C. J. Li, “Spectral Reflectance Reconstruction Based on Camera Raw RGB Using the Weighted Polynomial of Order 3 and the Wiener Estimation,” J. Opt. Soc. Am. A 25(9), 2286 (2008). [CrossRef]  

18. C. J. Li, G. H. Cui, and M. R. Luo, “The accuracy of polynomial models for characterizing digital cameras,” Proceedings of AIC Color 2003 Bangkok, Colour communication and management (2003), 166–170.

19. P. Urban, M. R. Rosen, and R. S. Berns, “Spectral Gamut Mapping Framework based on Human Color Vision,” 4th European Conference on Colour in Graphics, Imaging, and Vision and 10th International Symposium on Multispectral Colour Science, (Terrassa, Spain, June 9-13, 2008), pp. 548–553.

20. CIE 15:2018 Colorimetry, 4th ed. (Vienna, Austria, 2019).

21. F. H. Imai, R. S. Berns, and D. Tzeng, “A comparative analysis of spectral reflectance estimated in various spaces using a trichromatic camera system,” J. Imaging Sci. Tech. 44(4), 280–287 (2000).

22. D. Carreres-Prieto, J. T. García, F. Cerdán-Cartagena, and J. Suardiaz-Muro, “Performing Calibration of Transmittance by Single RGB-LED within the Visible Spectrum,” Sensors 20(12), 3492 (2020). [CrossRef]  

23. F. H. Imai, M. R. Rosen, and R. S. Berns, “Comparative Study of Metrics for Spectral Match Quality,” in First European Conference on Colour in Graphics, Imaging and Vision,(Poitiers, France, 2002), pp. 492–496.

24. J. Hernández-Andrés, J. Romero, and R L Jr. Lee, “Colorimetric and spectroradiometric characteristics of narrow-field-of-view clear skylight in Granada, Spain,” J. Opt. Soc. Am. A 18(2), 412–420 (2001). [CrossRef]  

25. (https://www.color.org/iccmax/download/DemoIccMAX-Testing-2.1.19.zip). It can be found in Testing/SpecRef/SixChanCameraRef.icc (and associated xml file).

26. P. Zhu and A. V. Knyazev, “Angles between subspaces and their tangents,” Journal of Numerical Mathematics 21(4), 325–340 (2013). [CrossRef]  

27. S. Le Moan, J. Blahová, P. Urban, and O. Norberg, “Five dimensions for spectral color management,” J. Imaging Sci. Technol. 60(6), 60501-1–60501-9 (2016). [CrossRef]  

Data Availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (4)

Fig. 1.
Fig. 1. Data flow chart for the training (top red dashed part), compression (middle green dashed part), and reconstruction (bottom blue dashed part) for the proposed ICS.
Fig. 2.
Fig. 2. Spectral images from an oil painting for performance testing
Fig. 3.
Fig. 3. ICC sensors (a) and sensors (b) for the proposed ICS.
Fig. 4.
Fig. 4. Standard deviation versus wavelength for each ICSs under each of the test datasets. Same colored solid and dotted curves indicate the associated method was trained using the Munsell dataset and test dataset respectively.

Tables (10)

Tables Icon

Table 1. Spectral accuracy in terms of average (Ave), worst (Worst), and median (Med) values of the RMSE and GFC measures for each of the methods considered using the NCS test dataset and trained by the Munsell reflectance dataset (grey background) and NCS test dataset (white background).

Tables Icon

Table 2. Colorimetric accuracy in terms of average (Ave), worst (Worst), and median (Med) values of CIEDE2000 colour difference for each of the methods considered using the NCS test dataset with training by the Munsell reflectance dataset (grey background) and NCS test dataset (white background).

Tables Icon

Table 3. Idem to Table 1, except the test dataset being spectral image IM1

Tables Icon

Table 4. Idem to Table 2, except the test dataset being spectral image IM1

Tables Icon

Table 5. Idem to Table 1, except the test dataset being spectral image IM2

Tables Icon

Table 6. Idem to Table 2, except the test dataset being spectral image IM2

Tables Icon

Table 7. Idem to Table 1, except the test dataset being spectral image IM3

Tables Icon

Table 8. Idem to Table 2, except the test dataset being spectral image IM3

Tables Icon

Table 9. Performance summary for the proposed method tested using the NCS dataset, IM1,IM2,IM3 and trained using both the Munsell dataset and test dataset, respectively.

Tables Icon

Table 10. Reconstructed reflectance boundary checking summary for all ICSs tested using the NCS dataset, IM1, IM2, IM3 and trained using both the Munsell dataset (grey background) and test dataset (white background) respectively.

Equations (27)

Equations on this page are rendered with MathJax. Learn more.

t = W D 65 r
M = ( W D 65 ) T ( W D 65 ( W D 65 ) T ) 1
r b = r M t
W D 65 r b = W D 65 r W D 65 ( W D 65 ) T ( W D 65 ( W D 65 ) T ) 1 t = t t = 0
t P Q R = ( U 3 ) T r b
t L M S = W L M S r b
r ^ t = M t
W D 65 r ^ t = W D 65 M t = t
r ^ = r ^ t + r ^ b
r = i = 1 6 w i s i
t R G B = ( w 1 w 2 w 3 )
t W 2 P = ( w 1 w 2 P )
r r e s t = r i = 1 6 w i s i
P = ( r r e s t ) T h 1
t D 65 A = W D 65 A r
M W = K r ( W D 65 A ) T ( W D 65 A K r ( W D 65 A ) T ) 1
r ^ t = M W t D 65 A
r b = r M W t D 65 A
R b = U D V
r ^ b = U m q m
q m = ( U m ) T r b
q m = M P , k f t , k
M P , k = Q m ( F k ) T ( F k ( F k ) T ) +
r ^ = r ^ b + r ^ t
t 2 S I = W 2 S I r
r ^ = ( W 2 S I U 6 ) 1 t 2 S I .
S ( T E , T R ) = 1 d j = 1 d cos ( θ j )
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.