Residual image recovery method based on the dual-camera design of a compressive hyperspectral imaging system

Xinyu Liu; Zeqing Yu; Shuhang Zheng; Yong Li; Yong Li; Yong Li; Xiao Tao; Fei Wu; Qin Xie; Yan Sun; Chang Wang; Chang Wang; Chang Wang; Zhenrong Zheng; Zhenrong Zheng; Zhenrong Zheng

doi:10.1364/OE.459732

1. Introduction

Hyperspectral imaging technology is able to identify, analyze, and classify objects according to the spectral characteristics of different materials. Additionally, researchers applied hyperspectral imaging in many fields, such as face recognition [1], food safety [2], agriculture surveillance [3], and military reconnaissance [4]. However, traditional hyperspectral imaging methods, such as whiskbroom or pushbroom, require point-by-point or line-scanning of objects [5], which are time-consuming. Compressive hyperspectral imaging is a computational hyperspectral imaging technology that uses compressive sensing theory. It reconstructs the hyperspectral image of the object through the measurements of one or very few shots. This can significantly improve the measurement speed and overcome the tradeoff between spatial resolution, spectral resolution, light collection efficiency, and measurement time of traditional methods [6].

The compressive sensing theory says that if signal x is compressible by transform coding with a known transform, and the reconstruct procedure is nonlinear, the number of measurements n is able to be dramatically smaller than the signal size m [7]. Using compressive sensing theory, the sparse signals can be recovered with very few measurements. Researchers first applied it to the field of hyperspectral imaging in 2007 by D. Brady et al. [6]. They proposed a dual-disperser coded aperture snapshot spectral system. Also, they proposed a simplified single disperser coded aperture snapshot spectral imaging (CASSI) system in 2008 [8]. After that, many hyperspectral imaging systems have been proposed [9–16]. Some of these systems added an additional camera (grayscale or RGB camera) to provide side information for imaging [17–20]. For example, the dual-camera compressive hyperspectral imaging system (DCCHI) [18,19] which utilizes a dual camera design, has the following advantages [18,21]. First, the uncoded measurement of the additional camera greatly eases the reconstruction problem and improve the performance of CASSI, and this system is even competitive to multi-frame CASSI. Second, the dual came design enables higher light efficiency compared to CASSI using random binary coded apertures that blocks half of the incident light. Third, the DCCHI system maintains the snapshot advantage rather than multi shots required in multi-frame CASSI. Moreover, the side information provided by the additional camera helps develop more efficient reconstruction algorithm. However, the original paper only using the combined CASSI and additional camera measurements for reconstruction through general compressive sensing algorithm. The structural information of the object contained in additional camera measurement is not fully utilized. Researchers proposed some algorithms to make better use of the information provided by the additional camera in the DCCHI system [21–26]. There are also algorithms that attempt to use GPU for parallel computing to speed up the recovery of DCCHI for near real-time imaging [27,28].

To effectively use the information provided by the additional camera and improve the reconstruction quality, this paper proposes a residual image recovery method based on the DCCHI system (DCCHI-RIR). First, it constructs a small dictionary D from the information provided by the additional camera and uses the least-squares method to combine the information of the two cameras to jointly estimate the coefficient $\boldsymbol{\mathrm{\theta}}$ of the hyperspectral image under the dictionary D to obtain the estimated hyperspectral image f_estimated = D$\boldsymbol{\mathrm{\theta}}$. This process exploits the structural similarity between the additional camera measurement and hyperspectral image. The resulting estimated hyperspectral image retains most of the structural and spatial information of the object with fairly high accuracy. Next, it uses the CASSI measurement minus the component of estimated image to get the residual data. Then, compressive sensing theory is used to reconstruct the residual image. Finally, it adds the residual image to the estimated image to further reduce the error between the estimated and real image to obtain a more accurate final result. Furthermore, two DCCHI system structures are taken as examples to compare the simulation results between this method and existing methods. The results show that the DCCHI-RIR can obtain a better reconstruction quality than existing algorithms, and the proposed method is also capable of improving other types of compressive hyperspectral systems that are suitable for applying the dual-camera design, such as the grating-based [14] or single-pixel imaging based spectral imaging systems [29,30].

2. System configuration and method

2.1 System model

The DCCHI system adds a grayscale or RGB camera for uncoded measurement, which is in conjunction with coded CASSI measurement and can thus greatly ease reconstruction problem [18]. Figure 1 shows the DCCHI system used in this paper. The light from the scene is equally divided into two directions by the beam splitter. The light in one direction passes through the CASSI system placed in the sequence of objective lens, dispersive prism, coded aperture, relay lens, dispersive prism, and a grayscale camera. The light from the other direction is directly captured by the additional camera. The additional camera can be grayscale or RGB camera, which are two types of systems.

Fig. 1. Illustration showing the DCCHI system.

Download Full Size | PDF

By taking the DCCHI system with the grayscale camera as an example, this subsection introduces the sensing process. Let the three-dimensional (3D) hyperspectral data cube be f₀(x,y,λ), where x and y are the spatial coordinates, and λ is the spectral coordinate. Assuming that the beam splitter divides the light equally, the spectral intensity distribution before the coded aperture after dispersion is f₁(x,y,λ) 0.5f₀(x − ϕ(λ),y,λ), where ϕ(λ) donates the wavelength-dependent dispersion introduced by the dispersive prism. Assuming that the transmittance function of the coded aperture is T(x,y), the spectral density after the coded aperture is f₂(x,y,λ) f₁(x,y,λ)T(x,y). After the light passes through the second dispersive prism for the opposite dispersion to the first, the final spectral intensity distribution on the sensor becomes f₃(x,y,λ) f₂(x + ϕ(λ),y,λ). Donate Ω(λ) as the spectral response function of the CASSI detector. Combining the above formulas, the final measurement of the CASSI sensor is as follows:

(1)$$\textrm{g}(x,y) = \int {\varOmega (\lambda ){f_{_3}}(x,y,\lambda )} \textrm{d}\lambda = 0.5\int {\varOmega (\lambda ){f_{_0}}(x,y,\lambda )} T(x + \phi (\lambda ),y)\textrm{d}\lambda .$$

Then, the detector measurement for pixel (m, n) become:

(2)$${g_{mn}} = \int_{m\Delta }^{(m + 1)\Delta } {\int_{n\Delta }^{(n + 1)\Delta } {g(x,y)} } \textrm{d}x\textrm{d}y,$$

where Δ is the pixel size of the CASSI sensor. The distribution of dispersion is assumed to be linear: ϕ(λ)=α(λ−λ₀). Such an assumption is approximately true over the limited wavelength range studied; it can also be used in the nonlinear region by introducing some corrections [6]. Generally, it can be assumed that adjacent wavelength intervals are separated by a distance of one pixel on the coded aperture after dispersion. Considering x, y, and λ as discrete, the discrete indicators are m, n, and k respectively, and the discrete representation of the measurement of the pixel (m, n) is:

(3)$${g_{mn}} = 0.5\sum\limits_{k = 0}^{\textrm{B} - 1} {{\Omega _k}{f_{mnk}}{T_{\textrm{(}m + k\textrm{)}n}}} ,$$

where B is the total number of the hyperspectral bands, and Ω_k is the spectral response of the detector for the k th spectral band (k = 1,……, B). Suppose that the image of one wavelength band exhibits H rows and W columns. Arrange all g_mn and f_mnk into the detected 2D image G_CASSI (G_CASSI ∈ ℝ ^H × W) and the total 3D hyperspectral image F (F ∈ ℝ ^{H × W × B}), respectively. Donate g_CASSI (g_CASSI ∈ ℝ ^HW × 1) and f (f ∈ ℝ ^HWB × 1) as the vectorization of G_CASSI and F, respectively. Therefore, Eq. (1) can be expressed in the linear matrix form:

(4)$${{\mathbf g}_{{\bf CASSI}}} = {{\mathbf H}_{{\bf CASSI}}}{\mathbf f},$$

where H_CASSI (H_CASSI ∈ ℝ ^{HW × HWB}) is the observation matrix of the CASSI system.

Similarly, the measurement of the pixel (m, n) of the grayscale camera is given by

(5)$${g_{mn}} = 0.5\sum\limits_{k = 0}^{\textrm{B} - 1} {{\Omega _k}{f_{mnk}}} ,$$

and the corresponding linear matrix form is given by

(6)$${{\mathbf g}_{{\bf gray}}} = {{\mathbf H}_{{\bf gray}}}{\mathbf f}.$$

The earliest literature [18] that proposed DCCHI is to combine Eq. (4) and Eq. (6) for calculation. Let g = [ g_CASSI; g_gray], H = [ H_CASSI; H_gray]. The total detected signal g can be expressed as

(7)$${\mathbf g} = {\mathbf H\mathbf f}.$$

Then, total variation (TV) regularization [31] and two-step iterative shrinkage/thresholding (TwIST) algorithm [32] are adopted to reconstruct f:

(8)$$\mathop {\mathbf f}\limits^{\wedge} = \mathop {\arg \min }\limits_{\mathbf f} [{{{||{{\mathbf g} - {\mathbf{H}\mathbf{f}}} ||}^2} + \tau {\textrm{TV}(}{\mathbf f})} ],$$

where τ is the regularization coefficient. The TV term represents the smoothness of the hyperspectral image:

(9)$$\textrm{TV(}{\mathbf f}\textrm{) = }\sum\limits_k {\sum\limits_{m\textrm{,}n} {\sqrt {{{({f_{\textrm{(}m\textrm{ + 1)}nk}} - {f_{mnk}})}^2} + {{({f_{m\textrm{(}n\textrm{ + 1)}k}} - {f_{mnk}})}^2}} } } .$$

However, this method does not take full advantage of the information provided by the additional camera. Due to the smoothing effect of total variation, some spatial details will be failed to reconstruct. Therefore, this paper proposes the DCCHI-RIR method which is able to effectively utilize the similarity between the additional camera measurement and hyperspectral image. Thus, the spatial details can be preserved more and the reconstruction quality can be improved.

2.2 Proposed method for DCCHI system with grayscale camera

By taking the DCCHI system with the grayscale camera as an example, this subsection introduces the proposed DCCHI-RIR method. First, the way to obtain the estimated hyperspectral image is introduced. Let f_k be the vectorization data of the k th wavelength band (k = 1, 2, ……, B; f_k ∈ ℝ ^HW × 1). Then, f can be expressed as a concatenation of B f_k: f = [f₁;f₂;f₃;……;f_B]. Because the measurement of the grayscale camera is the sum of all spectral bands, the image captured by the grayscale camera contains the shape and structural information of the scene and demonstrates a certain structural similarity with the spectral image of each band. Therefore, g_gray is able to be used to estimate f_k. The simplest way to estimate f_k is f_k≈g_gray/B. Then, f can be simply expressed as f = [g_gray; g_gray; g_gray;……; g_gray]/B. However, this result is inaccurate. To estimate the f more accurately, the measurement of the CASSI sensor g_CASSI must also be used. First, B vectors from atom₁ to atom_B is constructed. Let the k th vector be atom_k (1 ≤ k ≤ B, atom_k ∈ ℝ ^HWB × 1). Then, these B vectors can be expressed as:

(10)$$\left\{ {\begin{array}{{c}} {{\mathbf {atom}_{\mathbf 1}} = \frac{1}{\textrm{B}}[{{{\mathbf g}_{{\bf gray}}};{\mathbf z};{\mathbf z};{\mathbf z};\ldots \ldots .;{\mathbf z}} ]}\\ {{\mathbf {atom}_{\mathbf 2}} = \frac{\mathbf 1}{\textrm{B}}[{{\mathbf z};{{\mathbf g}_{{\bf gray}}};{\mathbf z};{\mathbf z};\ldots \ldots .;{\mathbf z}} ]}\\ {{\mathbf {atom}_{\bf 3}} = \frac{1}{\textrm{B}}[{{\mathbf z};{z};{{\mathbf g}_{{\bf gray}}};{\mathbf z};\ldots \ldots .;{\mathbf z}} ]}\\{{\mathbf {atom}_{\bf B}} = \frac{1}{\textrm{B}}[{{\mathbf z};{\mathbf z};{\mathbf z};\ldots \ldots ;{\mathbf z};{{\mathbf g}_{{\bf gray}}}} ]} \end{array}} \right.$$

where z ∈ ℝ ^HW × 1 is a vertical vector whose all elements are zero. In Eq. (10), each atom_k consists of one g_gray and (B − 1) z. Then, a matrix D (D ∈ ℝ ^HWB × B) is constructed as the combination of all atom_k:

(11)$${\mathbf D} = [{{\mathbf {atom}_\textrm{1}},{\mathbf {atom}_\textrm{2}},\ldots \ldots .,{\mathbf {atom}_\textrm{B}}} ].$$

D can be seen as a small dictionary to express the estimated hyperspectral data. Suppose that:

(12)$${{\mathbf f}_{{\mathbf {estimated}}}} = {\mathbf D \mathbf \theta },$$

where f_estimated is the estimated hyperspectral images, and $\boldsymbol{\mathrm{\theta}}$ is the coefficient of the small dictionary D. The detected signal g can be approximated as the product of H and f_estimated: g≈Hf_estimated≈HD$\boldsymbol{\mathrm{\theta}}$. Therefore, the $\boldsymbol{\mathrm{\theta}}$ can be obtained by solve the overdetermined equation g = HD$\boldsymbol{\mathrm{\theta}}$. This overdetermined equation does not have exact solutions. However, a least-squares solution is able to be obtained by minimize the squared error between HD$\boldsymbol{\mathrm{\theta}}$ and g. The least-square solution is: $\boldsymbol{\mathrm{\theta}}$ = [(HD)^T (HD)]⁻¹ (HD)^T g. Therefore, a more accurate version of f_estimated is given by

(13)$${{\mathbf f}_{{\mathbf {estimated}}}} = {\mathbf D}{[{({\mathbf H\mathbf D})^\textrm{T}}({\mathbf H\mathbf D})]^{ - 1}}{({\mathbf H\mathbf D})^\textrm{T}}{\mathbf g}.$$

In this paper, a patch-based method is adopted. The measurements are divided into overlapping small patches; each small patch is recovered by parallel computing on the GPU, and the overlapping parts are averaged. This reconstruction strategy can significantly improve the accuracy and speed when calculating the estimated hyperspectral images. To calculate Eq. (13), HD must be calculated first for each patch. H is a large sparse matrix with many zero elements, and D also exhibits many zero elements. To increase the speed of the program, transform the formula to remove the unneeded 0 by 0 multiplications:

(14)$$\left\{ {\begin{array}{{l}} {{\mathbf H\mathbf D} = [{\mathbf H_{\mathbf {CASSI}}};{\mathbf H \mathbf {gray}}]{\mathbf D}}\\ {{\mathbf H_\mathbf{CASSI}\mathbf D} = [\Omega 1{\mathbf C1}\cdot \frac{{{{\mathbf g}_{{\bf gray}}}}}{\textrm{B}},\ldots \ldots ,\Omega \textrm{B}{\mathbf CB}\cdot \frac{{{{\mathbf g}_{{\bf gray}}}}}{\textrm{B}}]}\\ {{\mathbf H\textrm {gray}\mathbf D} = \Omega 1\frac{{{{\mathbf g}_{{\bf gray}}}}}{\textrm{B}} + \ldots \ldots + \Omega \textrm{B}\frac{{{{\mathbf g}_{{\bf gray}}}}}{\textrm{B}} = (\Omega 1 + \ldots \ldots \Omega \textrm{B})\frac{{{{\mathbf g}_{{\bf gray}}}}}{\textrm{B}}} \end{array}} \right.,$$

where C_k represents the coded aperture that the k th spectral band passes through. Eq. (14) can be derived from the imaging process of the system, and each atom of the small dictionary D can be regarded as a hyperspectral image with B bands. As a result, the number of multiplications is significantly reduced by Eq. (14).

Although the estimated hyperspectral image demonstrates high accuracy, a residual between it and the ground truth still exists. If the residual can be recovered, the imaging accuracy can be further improved. Suppose that f_residual is the residual error between f_estimated and f: f_residual = f − f_estimated. To reconstruct f_residual, multiply both sides by H at the same time: Hf_residual = g − Hf_estimated. Let r be the residual data whose components of f_estimated has been subtracted from the detected signal: r = g − Hf_estimated. The final constraint that f_residual need to satisfy are given by

(15)$${\mathbf H}{{\mathbf f}_{{\mathbf {residual}}}} = {\mathbf r}.$$

Since Eq. (15) is a severely underdetermined equation, and f_residual is sparse in TV regularization, TwIST algorithm and TV regularization can be used to reconstruct f_residual:

(16)$${\hat{{\mathbf f}}_{{\mathbf {residual}}}} = \mathop {\arg \min }\limits_{{{\mathbf f}_{{\mathbf {residual}}}}} [{{{||{{\mathbf r} - {\mathbf H}{{\mathbf f}_{{\mathbf {{residual}}}}}} ||}^2} + \tau \textrm{TV(}{\mathbf f_\mathbf {residual}}\textrm{)}}].$$

Then, the final result is the sum of f_estimated and f_residual:

(17)$$\hat{{\mathbf f}} = {\hat{{\mathbf f}}_{{\mathbf {residual}}}} + {{\mathbf f}_{{\mathbf {estimated}}}}.$$

In summary, the DCCHI-RIR is divided into two steps. The first step is to combine the measurements from the additional camera and CASSI sensor to quickly obtain an estimated hyperspectral image with high accuracy using Eq. (13). The second step is to reconstruct the residual hyperspectral image using the compressive sensing theory according to Eq. (16). Then, the residual hyperspectral image is added to the estimated hyperspectral image to obtain a more accurate final result. It is worth noting that the accuracy of the estimated hyperspectral image decreases when the spectral image of each band greatly differs from the grayscale image. However, due to the calculation of the residual, DCCHI-RIR can still guarantee that the reconstruction quality is similar to the general compressive sensing algorithm.

2.3 Proposed method for DCCHI system with RGB camera

This subsection introduces DCCHI-RIR by taking the DCCHI system with RGB camera as an example. The DCCHI system with RGB camera is also shown in Fig. 1. Let the detected RGB image be G_rgb (G_rgb ∈ ℝ ^{H × W × 3}). Then, the sensing process of the RGB camera can be expressed as:

(18)$${{\mathbf g}_{{\bf rgb}}} = {{\mathbf H}_{{\bf rgb}}}{\mathbf f},$$

where g_rgb is the vectorization of G_rgb (g ∈ ℝ ^3HW × 1), and H_rgb is the observation matrix of the RGB branch.

Let g = [ g_CASSI; g_rgb ] and H = [ H_CASSI; H_rgb ]. Eq. (7) can be obtained by combining Eq. (4) and Eq. (18). For the DCCHI system with an RGB camera, the way for constructing D is different. Let the images of the red, green, and blue channel of the RGB camera be G_r, G_g, and G_b, respectively (G_r, G_g, and G_b ∈ ℝ ^H × W). Here g_r, g_g, and g_b are their vectorization, respectively (g_r, g_g, and g_b∈ ℝ ^HW × 1). Obviously, g_rgb = [g_r, g_g, g_b]. Also, the relationship between the measurement of RGB camera and hyperspectral image can be expressed as:

(19)$$[{{{\mathbf g}_{\bf r}},{{\mathbf g}_{\bf g}},{{\mathbf g}_{\bf b}}} ]= [{{\mathbf f}_{\bf 1}},{{\mathbf f}_{\bf 2}},\ldots \ldots ,{{\mathbf f}_{\bf B}}]{\mathbf A},$$

where A ∈ ℝ ^B × 3 is the spectral response function of the RGB camera. Donate the (i, j) th element of A as a_i_,j (i = 1, 2, ……, B; j = 1,2,3); each spectral band can be estimated as

(20)$${\hat{{\mathbf f}}_{\mathbf k}} \mathbf \approx {\textrm{a}_{k\textrm{,1}}}{{\mathbf g}_{{rn}}} + {\textrm{a}_{k\textrm{,2}}}{{\mathbf g}_{\mathbf{gn}}} + {\textrm{a}_{k\textrm{,3}}}{{\mathbf{g}}_{\mathbf{bn}}}$$

where g_rn, g_gn, and g_bn are the normalized form of g_r, g_g, and g_b, respectively. The B atoms can be expressed as

(21)$$\left\{ {\begin{array}{{c}} {{\mathbf{ato}}{{\mathbf{m}}_{\mathbf{1}}} = \left[ {{{{\mathbf{\hat{f}}}}_{\mathbf{1}}};{\mathbf{z}};{\mathbf{z}};{\mathbf{z}};.......;{\mathbf{z}}} \right]} \\ {{\mathbf{ato}}{{\mathbf{m}}_{\mathbf{2}}} = \left[ {{\mathbf{z}};{{{\mathbf{\hat{f}}}}_{\mathbf{2}}};{\mathbf{z}};{\mathbf{z}};.......;{\mathbf{z}}} \right]} \\ {{\mathbf{ato}}{{\mathbf{m}}_3} = \left[ {{\mathbf{z}};{\mathbf{z}};{{{\mathbf{\hat{f}}}}_{\mathbf{3}}};{\mathbf{z}};.......;{\mathbf{z}}} \right]} \\ {......} \\ {{\mathbf{ato}}{{\mathbf{m}}_B} = \left[ {{\mathbf{z}};{\mathbf{z}};{\mathbf{z}};.......;{\mathbf{z}};{{{\mathbf{\hat{f}}}}_{\mathbf{B}}}} \right]} \end{array}} \right..$$

It is worth noting that Eq. (21) is not the only way to construct atom_k, but in our simulations, this way demonstrates the best results. Then, the dictionary D can be constructed as Eq. (11). When calculating the estimated hyperspectral image, the calculation of HD is still able be simplified as follows:

(22)$$\left\{ {\begin{array}{{l}} {{\mathbf{HD}} = [{\mathbf{H_{\mathbf CASSI}}};{\mathbf{H_{rgb}}}]{\mathbf{D}}} \\ {{\mathbf{H_{\mathbf CASSID}}} = [\Omega 1{\mathbf{C_1}}\cdot{{{\mathbf{\hat{f}}}}_{\mathbf{1}}},......,\Omega_{\text{B}}{\mathbf{C_B}}\cdot{{{\mathbf{\hat{f}}}}_{\mathbf{B}}}]} \\ {{\mathbf{HrgbD}} = [{{\text{a}}_{{\text{1}},1}}{{{\mathbf{\hat{f}}}}_{\mathbf{1}}},......,{{\text{a}}_{{\text{B}},1}}{{{\mathbf{\hat{f}}}}_{\mathbf{B}}};{{\text{a}}_{{\text{1}},2}}{{{\mathbf{\hat{f}}}}_{\mathbf{1}}},.....,{{\text{a}}_{{\text{B}},2}}{{{\mathbf{\hat{f}}}}_{\mathbf{B}}};{{\text{a}}_{{\text{1}},3}}{{{\mathbf{\hat{f}}}}_{\mathbf{1}}},.....,{{\text{a}}_{{\text{B}},3}}{{{\mathbf{\hat{f}}}}_{\mathbf{B}}}]} \end{array}} \right..$$

The remaining steps of DCCHI-RIR are the same as Eq. (15-17).

The DCCHI-RIR algorithm flow of the two systems is shown in Algorithms 1 and 2. The framework schematic of the DCCHI-RIR is shown in Fig. 2. It is worth mentioning that the algorithm flow and framework schematic are for each overlapping small patch. After recovering each small patch, adding them up and averaging the overlapping part value are necessary.

oe-30-11-20100-i001

Fig. 2. Framework schematic of DCCHI-RIR.

Download Full Size | PDF

3. Results and discussions

3.1 Results comparison for DCCHI system with grayscale camera

To evaluate the effectiveness of DCCHI-RIR, Natural Scenes 2015 dataset [33] is used. This dataset contains 30 hyperspectral images with a wavelength range of 400-720 nm sampled at an interval of 10 nm, and each pixel value represents spectral radiance in Wm⁻²sr⁻¹nm⁻¹. In this paper, the wavelength from 400 to 700 nm has been used, and six hyperspectral images are randomly selected as our test images. The resolution of the selected test images is set to 520×520×31. All test hyperspectral images are scaled to the interval from 0 to 1 for easier comparison.

Additionally, three quantitative metrics are employed to evaluate the reconstructed hyperspectral images, including spectrum angular mapper (SAM) [34], mean peak signal-to-noise ratio (M-PSNR), and mean structural similarity (M-SSIM). SAM is a metric to measure the spectral accuracy whose value range is from 0 to 1. The smaller the SAM, the more accurate the spectrum. M-PSNR is a metric to calculate the PSNR of each spectral band and finally take the average value. PSNR is the image evaluation index of the overall recovery effect; a large PSNR value represents a good recovery effect. M-SSIM is used to calculate the SSIM of each spectral band first and then calculate the mean value. SSIM is an indicator to measure the similarity of the image structure, and its value range is from 0 to 1; a high SSIM value means less distortion of the restored image.

Figure 3 shows the 15th band (540-550 nm) of the estimated and residual images of six 100×100 regions. The estimated images are calculated according to Eq. (13). To examine the properties of the residual images to be reconstructed, the residual images in Fig. 3 are not reconstructed according to Eq. (16) but obtained by subtracting the estimated hyperspectral images from the original hyperspectral images. The estimated hyperspectral images obtained in the first step of DCCHI-RIR preserve most of the spatial details of the original hyperspectral images and demonstrate high accuracy. As can be seen from results in Fig. 3, the estimated hyperspectral images in Fig. 3(b1-b6) are very similar with the original hyperspectral images in Fig. 3(a1-a6) when being visually observed. Additionally, the quality metrics in Fig. 3(b1-b6) also verify this similarity. Comparing Fig. 3(a1-a6) with Fig. 3(c1-c6), the residual hyperspectral images exhibit smaller interval ranges than the original hyperspectral images. While the normalized residual images can be obtained through normalizing the differences of residual images and their minimum values, and the corresponding images shown in Fig. 3(c1-c6) still have similar shapes to the original images. In addition, the TVs of normalized residual images are also close to the original images, and as a result it is reasonable to use TV regularization for reconstructing the residual hyperspectral images.

Fig. 3. Visual results of the estimated and normalized residual hyperspectral images in one spectral band (500-510 nm) from six regions of the test hyperspectral images. The TVs and the intervals of the original hyperspectral images are listed in (a1-a6). The quality metrics (SAM, M-PSNR and M-SSIM) of the estimated images are listed in (b1-b6). The TVs of the normalized residual images and the intervals of the original residual images are listed in (c1-c6).

Download Full Size | PDF

The proposed method is compared with other methods for DCCHI system with the grayscale camera: TwIST with TV regularization (DCCHI-TwIST) [18], dictionary-based method (DCCHI-DBR) [25], and adaptive nonlocal sparse representation method (DCCHI-ASNR) [26]. The dictionary for DCCHI-DBR is trained using the remaining 24 datasets. First, the 24 datasets are divided into several 5×5×31 small patches. Then, some patches with all zero elements or very little total variance are removed. The remaining patches make up the training set and are used to train the DBR dictionary using K-SVD algorithm. The principal component analysis (PCA) sub-dictionaries for DCCHI-ASNR method are also trained using the above training sets. First, the training set is divided into 70 clusters using K-means clustering method. Then, each cluster is used to train a PCA sub-dictionary. Fig. 4 shows our strategy for dividing the overlapping patches. The detected image is divided into four types of overlapping patches of grey, green, blue, and red. There are three types of regions in Fig. 4, marked with horizontal, vertical, and diagonal lines. The first area is covered only by one type of rectangle. The second area is covered by two types of rectangles, and the final result is calculated by dividing by 2. The third area is covered by all four types of rectangles, and the final result is calculated by dividing by 4. It is worth mentioning that the patch size must match the condition of h × w ≥ B for the DCCHI-RIR, where h and w are the height and width of the patches, because the inverse of the matrix [(HD)^T(HD)] needs to be obtained when calculating Eq. (13), and it is necessary to ensure that the matrix is full rank. Considering the balance of reconstruction quality and calculation speed, the patch size for DCCHI-TwIST and DCCHI-RIR is 10×10×31.

Fig. 4. Strategy for dividing the overlapping patches.

Download Full Size | PDF

Table 1 presents the reconstruction results of different methods. For all hyperspectral images, the estimated hyperspectral images obtained in the first step of the DCCHI-RIR demonstrate higher accuracy than other methods. After adding the residual hyperspectral images obtained in the first step of the DCCHI-RIR, the accuracy of the final results is further improved. Because the six test images exhibit high nonlocal similarity, and many small patches are similar, the DCCHI-ASNR using the nonlocal similarity of hyperspectral images outperforms the other two methods, second only to the proposed method. The proposed method has an average SAM reduction of 0.0184, an average M-PSNR improvement of 3.62 dB, and an average M-SSIM improvement of 0.0582.

Table 1. SAM, PSNR, and SSIM comparison of different methods for DCCHI system with the grayscale camera

View Table | View all tables in this article

Fig. 5 shows the visual comparison of reconstructed images for different methods. The reconstructed images of the DCCHI-TwIST, DCCHI-DBR, and DCCHI-ASNR demonstrate different degrees of color distortion in Fig. 5(b1-e1). The magnified areas marked with the red boxes in Fig. 5(b1) and (b2) show that the reconstructed images of DCCHI-TwIST exhibit a blocking effect due to inaccurate reconstruction. Figure 5(b2-d2) exhibit some colored stripes in the magnified areas due to the color distortion. However, in Fig. 5(e1) (f1) and Fig. 5(e2) (f2), the estimated and reconstructed images of the DCCHI-RIR exhibit no block effect and color stripes, and the colors are almost the same as the original images.

Fig. 5. Visual comparison of reconstruction results using different methods. For ease of display, the entire 3D hyperspectral data are synthesized into an RGB image.

Download Full Size | PDF

Fig. 6 shows the grayscale images of the 15th spectral band (540–550 nm) of the reconstructed hyperspectral images. The reconstructed images of other methods in the red boxes in Fig. 6(b1-d1) demonstrate shape deformation. The results of the other methods in Fig. 6(c2) and (d2) exhibit some streaks and speckle noise. The estimated and reconstructed images of the DCCHI-RIR have almost the same shapes as the original images in Fig. 6(e1) and (f1), and are smoother without streaks and speckle noise in Fig. 6(e2) and (f2).

Fig. 6. Visual comparison of reconstruction results for the 15th spectral band (540-550 nm) using different methods.

Download Full Size | PDF

Figure 7 shows the recovered spectra at two randomly selected spatial points A and B marked in 7(a). Figure 7(b) and (c) are the relative spectra of points A and B. The reconstruction spectra of the proposed method (the black line) are closest to the ground truth (red line).

Fig. 7. Spectra of two points reconstructed by different methods.

Download Full Size | PDF

Furthermore, GPU is used for computation to verify the effect of different patch sizes on the accuracy and calculation speed of estimated hyperspectral images. The calculation is performed on NVIDIA 2080ti and Intel i7-9700K and the program is run on Matlab 2021a using its parallel computing tool. The running time required is obtained by averaging 10 runs. The size of the hyperspectral images is set to 280×280×31 based on the running memory of the GPU. The quality metrics of the DCCHI-RIR for different patch sizes are also listed in Table 2. The second step of DCCHI-RIR is implemented through CPU calculation. When the patch size changes from 7×7×31 to 20×20×31, the required running time for estimated images decreases gradually. However, all times are less than 40 ms. The frame rate for reconstructing the estimated hyperspectral image exceeds 25 frames per second. For the estimated image, the patch size of 8×8×31 demonstrates the best reconstruction quality. However, the final result of DCCHI-RIR method has the best quality when the patch size is 10×10×31. Fig. 8 shows the visual comparison of the estimated images for different patch sizes. The results of all patch sizes demonstrate good visual effects. Therefore, the method of obtaining estimated hyperspectral images can be used alone as a real-time and high accuracy reconstruction method.

Table 2. The effect of different patch sizes based on DCCHI system with grayscale camera

View Table | View all tables in this article

Fig. 8. Estimated hyperspectral image visual comparison at different patch sizes based on the DCCHI system with grayscale camera. The total image size is 280×280×31. (a1-f1) are the synthesized RGB images. (a2-f2) are the grayscale images of the 15th band (540-550 nm) of the hyperspectral image.

Download Full Size | PDF

3.2 Results comparison for DCCHI system with RGB camera

In this paper, the RGB sensor adopts the Foveon structure, and each pixel can detect the red, green, and blue in sequence [35]. If the RGB camera has the Bayer pattern sensor, the raw image should be implemented demosaicing to obtain the full image [19]. The DCCHI-TwIST and DCCHI-DBR mentioned above can also be applied to this system. Also, the DCCHI-RIR is compared with the patch-based fusion algorithm (PFusion) proposed by Wei He et al. [28]. This method takes advantage of the low-rank property of the hyperspectral image and can reconstruct the hyperspectral image with high accuracy using far less computational time. It is worth noting that the patch size influences its low-rank property. When the patch size is too small, the performance of PFusion decreases significantly. Therefore, the patch size of our proposed method is first set to 10×10×31 for comparison with DCCHI-TwIST and DCCHI-DBR. Then, the patch size is set to 20×20×31 to compare with PFusion. The results are shown in Tables 3. Table 3 shows that the proposed method outperforms DCCHI-TwIST and DCCHI-DBR when the pitch size is 10×10×31. At the same time, DCCHI-RIR demonstrates a similar performance with the PFusion method when the pitch size is 20×20×31. However, the advantage of DCCHI-RIR is that it does not require the low-rank characteristics of hyperspectral images and can achieve better results when the patch size is small.

Table 3. SAM, PSNR, and SSIM comparison of different methods for DCCHI design with RGB camera

View Table | View all tables in this article

Table 4 and Fig. 9 show the effect of different patch sizes on the accuracy and speed of estimated hyperspectral images. The running time in Table 4 is longer than that in Table 3 because the RGB camera captures more data than the grayscale camera, thus, requiring more computational time. Therefore, the accuracy of Table 4 improves more than that of Table 3. It can be seen that the computation time for the estimated images is about 50 ms. The quality metrics of the DCCHI-RIR for different patch sizes are also listed in Table 4. For the DCCHI system with RGB camera, the estimated image has the best quality when the patch size is 8×8×31, meanwhile the final result of the DCCHI-RIR has the best quality when the patch size is 20×20×31.

Table 4. The effect of different patch sizes based on DCCHI system with RGB camera

View Table | View all tables in this article

Fig. 9. Estimated hyperspectral image visual effects at different patch sizes based on the DCCHI system with RGB camera. The total image size is 280×280×31. (a1-f1) are the synthesized RGB images. (a2-f2) are the grayscale images of the 15th band (540-550 nm) of the hyperspectral image.

Download Full Size | PDF

3.3 Robustness and generalization ability of the proposed method

The noise interference is an unavoidable problem in real world. To investigate the robustness of DCCHI-RIR, the white Gaussian noise with noise level σ = 0.01, 0.04 and 0.1 is added to the measurements of CASSI sensor and the grayscale camera, where σ denotes the standard deviation of the write Gaussian noise. To begin with, the quality metrics of the reconstruction results for all the six test datasets are obtained, and then their mean values are calculated. Finally, the mean values of the quality metrics are shown in Table 5. As the noise level increases, the reconstruction quality of the estimated hyperspectral images and the final results of the DCCHI-RIR degrades slowly. For all the noise levels, the proposed method demonstrates the best performance. Therefore, the robustness of the DCCHI-RIR is reliable for most case.

In order to verify the generalization ability of the proposed method, six groups of hyperspectral images are chosen randomly in another hyperspectral dataset of ICVL [36], and simulations have been carried out to get the reconstruction results of the DCCHI-RIR on DCCHI system with grayscale camera. The visual comparison of reconstruction results and their quality metrics are illustrated in Fig. 10, which shows that the SAM of all six reconstructed images are smaller than 0.05, the M-PSNR are all larger than 31 dB, and the SSIMs are all larger than 0.95. These results indicate that the proposed method can realize reconstruction results with high quality for other hyperspectral datasets.

Table 5. Quality metrics of different reconstruction methods at different noise levels based on DCCHI system with grayscale camera.

View Table | View all tables in this article

Fig. 10. Visual comparison of reconstruction results by DCCHI-RIR of the six test hyperspectral images in the ICVL dataset. The quality metrics of SAM, M-PSNR and M-SSIM are also listed.

Download Full Size | PDF

4. Conclusion

In summary, this paper proposes a residual image recovery method based on the dual-camera compressive hyperspectral imaging system. The proposed method utilizes the structural similarity between the image of the additional camera and the original hyperspectral image, which can more effectively utilize the side information and improve the reconstruction quality of the hyperspectral image. The simulation results show that DCCHI-RIR demonstrates higher accuracy than some state-of-the-art methods based on DCCHI system. Additionally, the proposed method can also be applied to other compressive hyperspectral systems with dual-camera design.

Funding

Beijing Municipal Science and Technology Commission (Z201100004020012).

Disclosures

The authors declare no conflicts of interest.

Data Availability

Data underlying the results presented in this paper are not publicly available but may be obtained from the authors upon reasonable request.

References

1. V. Sharma and L. Van Gool, “Image-level classification in hyperspectral images using feature descriptors, with application to face recognition,” https://arxiv.org/abs/1605.03428.

2. J. Qin, M. S. Kim, K. Chao, D. E. Chan, S. R. Delwiche, and B.-K. Cho, “Line-scan hyperspectral imaging techniques for food safety and quality applications,” Appl. Sci. 7(2), 125 (2017). [CrossRef]

3. R. N. Sahoo, S. Ray, and K. Manjunath, “Hyperspectral remote sensing of agriculture,” Curr. Sci. 108(5), 848–859 (2015).

4. I. Makki, R. Younes, C. Francis, T. Bianchi, and M. Zucchetti, “A survey of landmine detection using hyperspectral imaging,” ISPRS Journal of Photogrammetry and Remote Sensing 124, 40–53 (2017). [CrossRef]

5. R. G. Sellar and G. Boreman, “Classification of imaging spectrometers for remote sensing applications,” Opt. Eng. 44(1), 013602 (2005). [CrossRef]

6. M. E. Gehm, R. John, D. J. Brady, R. M. Willett, and T. J. Schulz, “Single-shot compressive spectral imaging with a dual-disperser architecture,” Opt. Express 15(21), 14013–14027 (2007). [CrossRef]

7. D. L. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006). [CrossRef]

8. A. Wagadarikar, R. John, R. Willett, and D. Brady, “Single disperser design for coded aperture snapshot spectral imaging,” Appl. Opt. 47(10), B44–B51 (2008). [CrossRef]

9. A. A. Wagadarikar, N. P. Pitsianis, X. Sun, and D. J. Brady, “Video rate spectral imaging using a coded aperture snapshot spectral imager,” Opt. Express 17(8), 6368–6388 (2009). [CrossRef]

10. T. Sun and K. Kelly, “Compressive sensing hyperspectral imager,” in Computational Optical Sensing and Imaging (Optical Society of America, 2009), paper CTuA5.

11. Y. Wu, I. O. Mirza, G. R. Arce, and D. W. Prather, “Development of a digital-micromirror-device-based multishot snapshot spectral imaging system,” Opt. Lett. 36(14), 2692–2694 (2011). [CrossRef]

12. Y. August, C. Vachman, Y. Rivenson, and A. Stern, “Compressive hyperspectral imaging by random separable projections in both the spatial and the spectral domains,” Appl. Opt. 52(10), D46–D54 (2013). [CrossRef]

13. X. Lin, G. Wetzstein, Y. Liu, and Q. Dai, “Dual-coded compressive hyperspectral imaging,” Opt. Lett. 39(7), 2044–2047 (2014). [CrossRef]

14. X. Lin, Y. Liu, J. Wu, and Q. Dai, “Spatial-spectral encoded compressive hyperspectral imaging,” ACM Trans. Graph. 33(6), 1 (2014). [CrossRef]

15. H. Arguello and G. R. Arce, “Colored coded aperture design by concentration of measure in compressive spectral imaging,” IEEE Trans. on Image Process. 23(4), 1896–1908 (2014). [CrossRef]

16. M. A. Golub, A. Averbuch, M. Nathan, V. A. Zheludev, J. Hauser, S. Gurevitch, R. Malinsky, and A. Kagan, “Compressed sensing snapshot spectral imaging by a regular digital camera with an added optical diffuser,” Appl. Opt. 55(3), 432–443 (2016). [CrossRef]

17. X. Yuan, T.-H. Tsai, R. Zhu, P. Llull, D. J. Brady, and L. Carin, “Compressive hyperspectral imaging with side information,” IEEE J. Sel. Top. Sig. Process. 9(6), 964–976 (2015). [CrossRef]

18. L. Wang, Z. Xiong, D. Gao, G. Shi, and F. Wu, “Dual-camera design for coded aperture snapshot spectral imaging,” Appl. Opt. 54(4), 848–858 (2015). [CrossRef]

19. L. Wang, Z. Xiong, G. Shi, W. Zeng, and F. Wu, “Compressive hyperspectral imaging with complementary RGB measurements,” in 2016 Visual Communications and Image Processing (IEEE, 2016), pp. 1–4.

20. J. Hauser, M. A. Golub, A. Averbuch, M. Nathan, V. A. Zheludev, and M. Kagan, “Dual-camera snapshot spectral imaging with a pupil-domain optical diffuser and compressed sensing algorithms,” Appl. Opt. 59(4), 1058–1070 (2020). [CrossRef]

21. C. Tao, H. Zhu, P. Sun, R. Wu, and Z. Zheng, “Hyperspectral image recovery based on fusion of coded aperture snapshot spectral imaging and RGB images by guided filtering,” Opt. Commun. 458, 124804 (2020). [CrossRef]

22. Y. Xu, Z. Wu, J. Chanussot, and Z. Wei, “Hyperspectral Computational Imaging via Collaborative Tucker3 Tensor Decomposition,” IEEE Trans. Circuits Syst. Video Technol. 31(1), 98–111 (2021). [CrossRef]

23. N. Cheng, H. Huang, L. Zhang, and L. Wang, “Snapshot Hyperspectral Imaging Based on Weighted High-order Singular Value Regularization,” in 2020 25th International Conference on Pattern Recognition (ICPR, 2021), pp. 1267–1274.

24. Z. Liang, Y. Xu, L. Xiao, and Z. Wei, “Spatial-Spectral Total Variation Constrained Collaborative Tensor Regularization for Dual-Camera Compressive Hyperspectral Imaging,” in 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS (IEEE, 2021), pp. 3873–3876.

25. L. Wang, Z. Xiong, D. Gao, G. Shi, W. Zeng, and F. Wu, “High-speed hyperspectral video acquisition with a dual-camera architecture,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (IEEE, 2015), pp. 4942–4950.

26. L. Wang, Z. Xiong, G. Shi, F. Wu, and W. Zeng, “Adaptive nonlocal sparse representation for dual-camera compressive hyperspectral imaging,” IEEE Trans. Pattern Anal. Mach. Intell. 39(10), 2104–2111 (2017). [CrossRef]

27. S. Zhang, L. Wang, Y. Fu, and H. Huang, “Gpu assisted towards real-time reconstruction for dual-camera compressive hyperspectral imaging,” in Pacific Rim Conference on Multimedia (Springer, 2018), pp. 711–720.

28. S. Zhang, H. Huang, and Y. Fu, “Fast parallel implementation of dual-camera compressive hyperspectral imaging system,” IEEE Trans. Circuits Syst. Video Technol. 29(11), 3404–3414 (2019). [CrossRef]

29. A. Jerez, H. Garcia, and H. Arguello, “Single Pixel Spectral Image Fusion with Side Information from a Grayscale Sensor,” in 2018 IEEE 1st Colombian Conference on Applications in Computational Intelligence (IEEE, 2018), pp. 1–6.

30. H. Garcia, C. V. Correa, and H. Arguello, “Optimized Sensing Matrix for Single Pixel Multi-Resolution Compressive Spectral Imaging,” IEEE Trans. on Image Process 29, 4243–4253 (2020). [CrossRef]

31. A. Chambolle, “An algorithm for total variation minimization and applications,” Journal of Mathematical Imaging and Vision 20(1/2), 73–87 (2004). [CrossRef]

32. J. M. Bioucas-Dias and M. A. Figueiredo, “A New TwIst: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration,” IEEE Trans. on Image Process. 16(12), 2992–3004 (2007). [CrossRef]

33. S. M. Nascimento, K. Amano, and D. H. Foster, “Spatial distributions of local illumination color in natural scenes,” Vision Res. 120, 39–44 (2016). [CrossRef]

34. F. A. Kruse, A. Lefkoff, J. Boardman, K. Heidebrecht, A. Shapiro, P. Barloon, and A. J. Goetz, “The spectral image processing system (SIPS)—interactive visualization and analysis of imaging spectrometer data,” Remote Sensing of Environment 44(2-3), 145–163 (1993). [CrossRef]

35. P. Hubel, J. Liu, and R. Guttosch, “Spatial frequency response of color image sensors: Bayer color filters and Foveon X3,” Proc. SPIE 5301, 402–407 (2004). [CrossRef]

36. B. Arad and O. Ben-Shahar, “Sparse recovery of hyperspectral signal from natural RGB images,” in IEEE European Conference on Computer Vision (Springer, 2016), pp. 19–34.

Image number	Quality metrics	DCCHI-TwIST	DCCHI-DBR	DCCHI-ASNR	Estimated image	DCCHI-RIR
Img 1	SAM	0.2409	0.1559	0.1057	0.0733	0.0712
	M-PSNR	22.42	26.26	31.99	37.75	37.97
	M-SSIM	0.7383	0.8555	0.9316	0.9717	0.9729
Img 2	SAM	0.1469	0.1151	0.1126	0.0771	0.0734
	M-PSNR	30.31	32.81	33.61	36.67	37.07
	M-SSIM	0.9115	0.9418	0.9429	0.9723	0.9740
Img 3	SAM	0.2793	0.2307	0.2114	0.1559	0.1523
	M-PSNR	31.62	38.40	37.83	41.77	42.03
	M-SSIM	0.9244	0.9654	0.9692	0.9878	0.9885
Img 4	SAM	0.2420	0.1679	0.1495	0.1466	0.1412
	M-PSNR	32.95	35.92	35.44	37.15	37.50
	M-SSIM	0.9333	0.9642	0.9661	0.9724	0.9743
Img 5	SAM	0.1194	0.0816	0.0585	0.0545	0.0523
	M-PSNR	34.67	39.32	40.76	44.80	44.94
	M-SSIM	0.9510	0.9751	0.9852	0.9903	0.9906
Img 6	SAM	0.1433	0.1152	0.1007	0.0794	0.0781
	M-PSNR	40.96	44.08	44.47	46.18	46.31
	M-SSIM	0.9755	0.9852	0.9880	0.9925	0.9928
Average	SAM	0.1953	0.1444	0.1231	0.0978	0.0947
	M-PSNR	32.16	36.13	37.35	40.72	40.97
	M-SSIM	0.9057	0.9479	0.9638	0.9812	0.9822

Patch size		7×7×31	8×8×31	10×10×31	14×14×31	20×20×31
Estimated time/ms		33.562	30.2759	30.3345	28.8963	26.0918
Estimated frame rates		29.80	33.03	32.97	34.61	38.33
Estimated image	SAM	0.0783	0.0749	0.0765	0.0850	0.0963
	M-PSNR	34.66	34.97	34.80	33.81	32.80
	M-SSIM	0.9626	0.9666	0.9677	0.9642	0.9594
DCCHI-RIR	SAM	0.0776	0.0733	0.0720	0.0726	0.0728
	M-PSNR	34.72	35.11	35.24	34.83	34.44
	M-SSIM	0.9630	0.9674	0.9699	0.9701	0.9701

Image number	Quality metrics	DCCHI-TwIST	DCCHI-DBR	Estimated image (10)	DCCHI-RIR (10)	PFusion	Estimated image (20)	DCCHI-RIR (20)
Img 1	SAM	0.2033	0.1116	0.0716	0.0651	0.0647	0.0803	0.0632
	M-PSNR	24.29	32.32	38.15	39.12	39.56	36.87	39.46
	M-SSIM	0.7848	0.9280	0.9730	0.9769	0.9780	0.9697	0.9787
Img 2	SAM	0.1242	0.0828	0.0698	0.0517	0.0436	0.0827	0.0480
	M-PSNR	32.16	38.17	38.35	42.07	44.55	36.64	43.08
	M-SSIM	0.9335	0.9690	0.9785	0.9878	0.9917	0.9747	0.9898
Img 3	SAM	0.2430	0.2097	0.1517	0.1375	0.1399	0.1702	0.1356
	M-PSNR	33.77	41.17	42.61	44.39	44.64	41.48	44.99
	M-SSIM	0.9445	0.9793	0.9890	0.9919	0.9917	0.9855	0.9924
Img 4	SAM	0.2294	0.1591	0.1394	0.1199	0.1206	0.1571	0.1167
	M-PSNR	33.85	37.81	37.86	39.43	39.19	36.47	39.38
	M-SSIM	0.9400	0.9706	0.9753	0.9818	0.9808	0.9688	0.9823
Img 5	SAM	0.1038	0.0703	0.0512	0.0451	0.0382	0.0540	0.0417
	M-PSNR	36.79	41.58	45.76	47.20	48.14	45.83	48.19
	M-SSIM	0.9622	0.9814	0.9911	0.9927	0.9945	0.9916	0.9938
Img 6	SAM	0.1203	0.1022	0.0753	0.0670	0.0652	0.0753	0.0632
	M-PSNR	42.79	45.44	46.95	48.30	48.60	47.20	49.20
	M-SSIM	0.9822	0.9883	0.9935	0.9950	0.9953	0.9937	0.9956
Average	SAM	0.1707	0.1226	0.0932	0.0811	0.0787	0.1033	0.0781
	M-PSNR	33.94	39.42	41.61	43.42	44.11	40.75	44.05
	M-SSIM	0.9246	0.9694	0.9834	0.9877	0.9887	0.9807	0.9888

Patch size		7×7×31	8×8×31	10×10×31	14×14×31	20×20×31
Estimated time/ms		58.54	54.55	51.87	51.01	47.19
Estimated frame rates		17.08	18.33	19.28	19.60	21.19
Estimated image	SAM	0.0714	0.0679	0.0686	0.0751	0.0832
	M-PSNR	36.25	36.68	36.58	35.67	34.68
	M-SSIM	0.9701	0.9738	0.9750	0.9730	0.9698
DCCHI-RIR	SAM	0.0603	0.0537	0.0492	0.0465	0.0455
	M-PSNR	38.09	39.34	40.39	41.03	41.39
	M-SSIM	0.9776	0.9824	0.9857	0.9876	0.9883

noise	σ = 0.01			σ = 0.04			σ = 0.1
Quality metrics	SAM	M-PSNR	M-SSIM	SAM	M-PSNR	M-SSIM	SAM	M-PSNR	M-SSIM
DCCHI-TwIST	0.1953	32.16	0.9057	0.1959	32.15	0.9056	0.2096	32.04	0.9032
DCCHI-DBR	0.1444	36.13	0.9479	0.1445	36.13	0.9478	0.1502	35.97	0.9459
DCCHI-ASNR	0.1231	37.35	0.9638	0.1241	37.34	0.9637	0.1525	36.90	0.9571
Estimated image	0.0978	40.72	0.9812	0.0985	40.71	0.9811	0.1167	40.14	0.9782
DCCHI-RIR	0.0948	40.97	0.9822	0.0956	40.95	0.9821	0.1189	40.24	0.9784

Residual image recovery method based on the dual-camera design of a compressive hyperspectral imaging system

Abstract

1. Introduction

2. System configuration and method

2.1 System model

2.2 Proposed method for DCCHI system with grayscale camera

2.3 Proposed method for DCCHI system with RGB camera

3. Results and discussions

3.1 Results comparison for DCCHI system with grayscale camera

3.2 Results comparison for DCCHI system with RGB camera

3.3 Robustness and generalization ability of the proposed method

4. Conclusion

Funding

Disclosures

Data Availability

References

Data Availability

Cited By

Figures (10)

Tables (5)

Equations (22)

Optics Express