Open Access
Add to BibSonomy
Abstract
Computational photography has been striving to capture the spectral information of the dynamic world in the last few decades. However, due to the curse of dimensionality between the 3D spectral images and the 2D imaging sensors, light-blocking components, e.g., bandpass filters or coded apertures, have to be used in spectral imaging systems to project the 3D signal into 2D measurements selectively. Accordingly, computational reconstruction is integrated to recover the underlying 3D signal. Thus, these systems suffer from low light efficiency and high computational burden, both of which cannot afford dynamic spectral imaging. In this paper, we propose a novel snapshot spectral imaging system that can dynamically capture the spectral images. The system is composed of a lens array, a notch filter array, and a monochrome camera in principle. Incoming light beams from the scene are spatially multiplexed by the lens array, spectrally mixed by the notch filter array, and captured by the monochrome camera. The two distinct characteristics, i.e., spatial multiplexing and spectral mixing, guarantee the advantages of low computational burden and high light efficiency, respectively. We further build a prototype system according to the imaging principles. The system can record two kinds of images in a single snapshot: bandstop multispectral images and a panchromatic image, which are used jointly to recover the bandpass multispectral images at few computational costs. Moreover, the proposed system is friendly with spectral super-resolution, for which we develop a theoretical demonstration. Both simulations and experiments are conducted to verify the effectiveness of the proposed system.
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Spectral imaging perceives and understands the real dynamic world with densely sampled spectral signatures in each spatial point [1]. The obtained spectral image delineates a detailed representation of the objects and lighting in the scene, which has been demonstrated to be beneficial to a profusion of computer vision tasks such as remote sensing [2], face recognition [3] and medical diagnosis [4]. The spectral image is three-dimensional (3D) by nature, i.e., two spatial dimensions and one spectral dimension, while existing imaging sensors are 1D linear arrays or 2D plane arrays. Therefore, the core problem in spectral imaging is how to deal with the curse of dimensionality between the 3D spectral images and the 2D imaging sensors [5,6]. Conventional spectrometers generally scan the scene along either the spatial dimension or the spectral dimension and need multiple exposures to capture a full spectral image. Thus, these systems are unsuitable for measuring dynamic scenes.
Computational photography has been striven to capture the spectral information of the dynamic world in the last few decades [7–12]. Under the efforts, snapshot spectral imagers have been developed through the advances of newly optics and increasingly computational power. Snapshot spectral imaging techniques can be mainly divided into two categories: direct imaging and computational imaging [13–15]. The most representative direct imaging systems include the thin observation module by bound optics (TOMBO) [16,17], the image replicating imaging spectrometer (IRIS) [18], the prism-mask imaging spectrometer (PMIS) [19,20], the optically replicating and remapping imaging spectrometer (ORRIS) [21]. These systems obtain high temporal resolution at the cost of some spatial and spectral resolutions. Due to the usage of light-blocking optical components such as bandpass filter, polarizer, occlusion mask or continuous variable filter to selectively transmit the light, direct imaging category usually suffers from severe light efficiency loss. Instead, computational imaging systems turn to obtain a set of coded projections of the 3D signal from which the spectral images can be reconstructed which is grounded on the rapid advances of newly optics and computational power. Representative computational imaging systems include the computed tomographic imaging spectrometry (CTIS) [13], the coded aperture snapshot spectral imagers (CASSI) [14,22], the spatial-spectral compressive spectral imaging (SSCSI) [15]. Similarly, these systems suffer from low light efficiency due to the usage of coded apertures. Further, this kind of technique needs to employ computation-intensive post-processing to reconstruct spectral images [23–25], which is unable to be carried out for real-time spectral image acquisition and thus precludes their applications from time-crucial tasks.
By delving into the existing spectral imaging systems, our key observation is that, for dynamic spectral imaging, the potential technique should meet two primary conditions: high light efficiency and low computational complexity. From the perspective of hardware design, the acquisition speed of images is mainly limited by the light efficiency. Only high light efficiency the hardware system has, the high frame imaging rate can be ensured. From the perspective of computational processing, the computational complexity should be as low as possible to warrant the real-time acquisition of spectral images.
In this paper, we propose a new snapshot spectral imaging system that can dynamically capture the spectral images. The key idea of the proposed system is based on a simple but not widely noticed phenomena, i.e., a notch filter passes the light in most spectrum unchangeably, but blocks those in a specific narrow range, which is the opposite of a bandpass filter. Principally, our system is composed of a lens array, a notch filter array, and a monochrome camera. As shown in Fig. 1, the incoming light beams from the scene are spatially multiplexed by the lens array, spectrally mixed by the notch filter array and captured by the monochrome camera. The spectral mixing introduced by the notch filters guarantees the advantage of high light efficiency. We further build a prototype system according to the basic principles. The system captures two kinds of images in a single snapshot: bandstop multispectral images, and a panchromatic image. With a straightforward minus operation, the bandpass multispectral images can be recovered from the captured images at few computational costs. Moreover, the proposed system is friendly with the spectral super-resolution owning to the high light efficiency, for which we develop a theoretical demonstration. To the best of our knowledge, it is the first time to make use of the notch filters to achieve high light efficiency snapshot spectral imaging. Both simulations on synthetic data and experiments on real captured data are conducted to evaluate the feasibility and effectiveness of the proposed system.
We summarize the main contributions as follows:
- • We proposed a high light efficiency multispectral imaging system via spatial multiplexing with a lens array and spectral mixing with notch filters.
- • We developed a spectral demixing method to recover multispectral images from spectral mixed measures assisted with an extra panchromatic image in real time.
- • We proved theoretically our system is friendly for spectral super-resolution.
- • We verified our method both on simulations and experiments with a built prototype.
2. System principles
We first analyze the light efficiency of our system, then formulate the system imaging model, and finally introduce the spectral super-resolution method.
2.1 Light efficiency analysis
The schematic of our system is shown in Fig. 1. Principally, our system is composed of a lens array, a notch filter array, and a monochrome camera. The incoming light beams from the scene are focused on the image plane, on which a field stop is placed to limit the field of view. A collimating lens is set behind the field stop to parallelize the light. The following notch filter array, consisting of several notch filters with different center wavelengths, can optionally transmit the spectrum, while the center of the notch filter array is equipped with no notch filter to allow light in all spectrum to pass. Finally, the lens array focuses the light going through each notch filter onto the monochromatic sensor separately. In this manner, our system can capture multiple bandstop spectral images and one panchromatic image.
Among various system design principles, light efficiency is a key factor that determines the performance of the snapshot spectral imaging. As the representative work in this direction, we evaluate the light efficiency of TOMBO [16], PMIS [20], ORRIS [21], IRIS [18], CTIS [13], CASSI [14], and SSCSI [15] along with our system. Specifically, we measure the light efficiency by the overall light transmission percentage of the whole system, which is mainly determined by the light-blocking elements in the optical path. Note that we only consider the ideal optical elements here, which may have a certain deviation in practice. To capture a spectral image with $\Lambda$ spectral bands, the light efficiency of different systems is summarized in Table 1. TOMBO uses bandpass filters which only transmit single narrow spectral band, therefore the light efficiency is $1/\Lambda$. ORRIS contains a continuous variable filter, of which a line can only transmit one spectral band, thus with $1/\Lambda$ light efficiency. PMIS incorporates an occlusion mask that down samples the spatial resolution by a factor of $\Lambda$ to obtain the desired spectral resolution, and thus only has a $1/\Lambda$ light throughput. IRIS utilizes a polarizer, waveplates and Wollaston polarizers to simultaneously obtains several sub images, with about 1/2 light efficiency loss due to the polarizer. CTIS suffers from $1/3$ light efficiency due to the inefficiencies in dispersion grating. CASSI and SSCSI involve one binary coded aperture with $1/2$ light transmission. Currently, the best performance in light efficiency can only reach $1/2$, e.g., IRIS, CASSI and SSCSI. By contrast, our system achieves light efficiency with $(\Lambda -1)/\Lambda$, since the solely light-blocking component in our system is the notch filters. Such high light efficiency enables us to capture spectral images in a fast manner. For example, the acquisition speed of our system is $\Lambda -1$ times of that of the systems using bandpass filters.
Table 1. Light efficiency comparisons of typical snapshot spectral imaging systems. Assume the spectral image is with $\Lambda$ spectral bands.
For a spectral imaging system, the tradeoff between spatial, spectral and temporal resolutions is always unavoidable. The proposed system, concentrating on the issue of light efficiency and temporal resolution, takes advantage of the lens array to divide the field, which results in necessarily reductions of spatial and spectral resolutions. Hence the system is more suitable for time crucial application scenarios naturally. It should be noticed that the number and the center wavelength of the notch filters can be adjusted flexibly according to the real application. Additionally, some super-resolution algorithms can be utilized such as spatial bicubic interpolation and spectral basis based methods [26]. As a result, the tradeoff between the spatial, spectral and temporal resolutions could be greatly relieved.
2.2 System imaging model
Denote a hyperspectral image as $F\in \mathbb {R}^{M\times N \times \Lambda }$, where $M$ and $N$ index the pixel positions and $\Lambda$ indexes the spectral channels. Suppose there are $I$ notch filters in the system. The bandstop multispectral images $S^{i}$ obtained through the $i_{th}$ notch filter can be expressed as
where $T_\lambda ^{i}$ represents the transmittance function of the $i_{th}$ notch filter and $F_\lambda$ denotes the $\lambda _{th}$ spectral band of the scene. Note that we ignore the quantum efficiency of the sensor for simplicity.The key idea of the proposed system is based on a simple but not widely noticed phenomena, i.e., a notch filter passes the light in most spectrum unchangeably, but blocks those in a specific narrow range, which is the opposite of a bandpass filter. We show the transmittance functions of several real notch filters in Fig. 2(a). For comparison, we also show the transmittance functions of several real bandpass filters used in conventional spectral imagers in Fig. 2(b). It can be seen that the notch filter allows the most portion of the light to transmit, while the bandpass filter only allows a small portion. Ideally, the transmittance rate of the notch filter should be as high as $100\%$ for the transmissive spectrum. However, due to the imperfection of the manufacturing technique, the transmittance rate actually is a constant $t_i$ in the interval of $[92\%, 96\%]$. Mathematically, the transmittance function of the $i_{th}$ notch filter $T_\lambda ^{i}$ can be expressed as
Fig. 2. Transmittance curves of different filters and their corresponding imaging model in matrix forms. (a) Notch filter; (b) Bandpass filter; (c) "Null filter" for the panchromatic image.
By substituting Eq. (2) into Eq. (1), the bandstop mlutipsectral images can be expressed as
Similarly, the panchromatic image can be formulated as the light going through a "null filter" In total, our system can capture $I$ bandstop multispectral images $S^{i}$ and one panchromatic image $P$. A captured image is shown in Fig. 1. Finally, with a straightforward minus operation, we can obtain $i_{th}$ bandpass multispectral images as Note that $\epsilon (i)$ means the stopband of the $i_{th}$ notch filter. As such, we can obtain the bandpass multispectral images by using bandstop multispectral images and a panchromatic image, most importantly, at few computational costs.2.3 Spectral super-resolution
Let us take a further step to understand the spectra. It has been demonstrated that the spectra of one pixel in natural scene can be decomposed as the linear combination of a small number of distinct spectral basis [27,28]. Thus, for the point $(m,n)$ in the scene, its spectra vector $F(m,n) = [F_1(m,n) \cdots ,F_{\lambda} (m,n), \ldots , F_{\Lambda} (m,n)]^{\intercal}$ can be written as
where $B \in \mathbb {R}^{\Lambda \times K}$ denotes the $K$ spectral basis and $\alpha (m,n) \in \mathbb {R}^{K}$ is the coefficient vector for point $(m,n)$. Then, according to Eq. (1), the spectra $S(m,n) = [S^{1}(m,n),\ldots ,S^{i}(m,n),\ldots ,S^{I}(m,n)]^{\intercal }$ of the bandstop multispectral image can be written as Here $T \in \mathbb {R}^{I\times \Lambda }$ is the system imaging model in matrix form, whose structure is illustrated in Fig. 2(a). Substituting Eq. (6) into Eq. (7), we can obtain Finally, by introducing a spectra smoothness constraint, the hyperspectral image is reconstructed by solving the following problem,By jointly considering the panchromatic image, the spectral super-resolution problem is further changed as
3. Simulations on synthetic data
Our system can directly output multispectral images as a straightforward minus operation. Thus we will directly conduct the experiments on real captured data to test the performance of our system on multispectral imaging. Here we focus on the simulations on synthetic data compared with other imaging methods. Firstly, We compare the proposed system against a direct spectral imaging system, a similar one equipped with bandpass filters which could be regard as an improved version for TOMBO, to verify the effectiveness of the spectral super-resolution of our system. Then we compare the proposed system with a computational imaging method, CASSI, to prove the superiority of reconstruction complexity.
3.1 Compared with direct imaging methods
For a comprehensive evaluation, we conduct simulations on two public spectral datasets, i.e., the CAVE dataset [29] and the ICVL dataset [30]. Both datasets contain HSIs with 31 spectral bands and 10nm spectral accuracy from $400nm$ to $700nm$. CAVE dataset contains 31 indoor scenes including toys, foods, oil paintings and so on, captured by a generalized assorted pixel camera which uses a complex array of color filters. ICVL dataset contains 201 HSIs which are acquired using a Specim PS Kappa DX4 hyperspectral camera and a rotary stage for spatial scanning. These images are captured from a variety of urban (residential/ commercial), suburban, rural, indoor and plant-life scenes. Both the basic configurations of the two spectral cameras and the contents of these datasets are distinct different, therefore the property of the HSIs from the two different datasets are diverse and heterogeneous.
The distinguishing feature of our system is the integration of the bandstop notch filters, which leads to high light efficiency, as we discussed in Sec. 2. Here, we compare our system against the one equipped with bandpass filters. By incorporating the effect of the panchromatic image, four different system configurations are compared for spectral super-resolution, i.e., (a) bandpass multispectral images (BP), (b) bandpass multispectral images together with a panchromatic image (BP+Pan.), (c) bandstop multispectral images (BS), (d) bandstop multispectral images together with a panchromatic image (BS+Pan.). The center wavelengths of both bandstop and bandpass filters are $488nm$, $532nm$, $561nm$, $594nm$, $642nm$, and $785nm$. Here, we use eight spectral bases by following the principles in [28]. The noise variance is set to 0.1 for all systems. The parameter $\eta$ in Eq. (10) are optimally set for each system. The peak signal to noise ratio (PSNR), structure similarity (SSIM), and the spectral angle mapper (SAM) are adopted as evaluation metrics. PSNR and SSIM indicate the spatial fidelity, and SAM indicates the spectral accuracy.
For quantitative comparisons, averaged results for the four methods on the two datasets, CAVE and ICVL, are shown in Table 2. More concretely, results for ten images from the CAVE dataset are listed in Table 3. The best performance for each dataset or image are highlighted in these tables. On the one hand, comparing the results between the bandstop filters and bandpass filters, we can see a significant gain in PSNR, SSIM, and SAM, which proves that the superiority of the high light efficiency system. On the other hand, the panchromatic image can further promote the performance, which demonstrates the friendliness of our system for spectral super-resolution.
Table 2. Reconstruction results (PSNR(dB)/SSIM/SAM) on datasets CAVE and ICVL for different methods. The results are averaged from all HSIs among the datasets.
Table 3. Reconstruction results (PSNR(dB)/SSIM/SAM) of ten HSIs from CAVE dataset for different methods. The best results are highlight in bold.
Figure 3 shows perceptual comparisons for two HSIs images, Stuffed toy from CAVE and Step from ICVL. We also provide the PSNR and SSIM values for each result image in parentheses. The last column shows the absolute spectral error of the white circles labeled on the ground truth images. The results from the bandpass filter-based system suffer from large noise. By contrast, our system can produce visually pleasant results with less noise and sharper edges, which is consistent with the numerical metrics.
Fig. 3. Reconstructed quality comparisons at 610nm for two images from CAVE and ICVL dataset. The average PSNR and SSIM values are presented in parentheses. The last column shows the absolute spectral error of the white circles labeled on the ground truth images. BS methods perform better than BP methods, and the reconstructions with a panchromatic image behave better than without that.
3.2 Compared with computational imaging methods
Here we compare our system with CASSI, one of the most representative computational imaging methods. Specifically, we conduct the comparison on CAVE dataset, of which the spatial size is $512 \times 512$ and the spectral band is $31$ from $400nm$ to $700 nm$. The standard deviation of the noise is 0.1. For the system CASSI, the binary transmission function of the coded aperture is generated as a random Bernoulli distribution with a probability 0.5. The reconstruction algorithm of CASSI is the commonly used two-step iterative shrinkage/thresholding (TwIST) [31]. The balance parameter is $0.8$ and the number of iterations is $200$, which are optimized with cross-validation. For our system, we assume 9 microlenses are integrated in the lens array, which introduces 9 spectral bands and a 3x3 spatial resolution tradeoff. Thus the captured image is with size of $171 \times 171 \times 9$, including 8 spectral mixed images and one panchromatic image. The 8 center wavelengths are $400nm$, $450nm$, $490nm$, $530nm$, $570nm$, $610nm$, $650nm$ and $700nm$, respectively. In the algorithm part, we reach spectral demixing from 9 bands to 31 bands by the proposed spectral basis based method and spatial super-resolution from $171 \times 171$ to $512 \times 512$.
Both CASSI and our system need to reconstruct spectral images with size of $512 \times 512 \times 31$ from the sensor measurement with pixel numbers as $512 \times 512$. The algorithms are run on a Window 10 64-bit system with Intel i7-7700 and 16GB RAM. Table 4 shows the average results of CASSI and our system on CAVE dataset. The better results are highlighted. From the table we can see, benefiting from the high light efficiency design, our system can always obtain better performance over CASSI. Particularly, the reconstruction time of our system is about three in a thousand of CASSI, demonstrating the low computation complexity of the proposed system.
Table 4. Reconstruction results comparisons (PSNR(dB)/SSIM/SAM/Time(second)) between the proposed system and CASSI on dataset CAVE. The results are averaged from all HSIs among the dataset.
In summary, the proposed system has been designed firstly for high speed multispectral imaging. By integrating with the super-resolution methods, it can obtain hyperspectral images with high spatial and spectral resolutions. Thus, the system could work in two modes: ultrafast mode and full resolution mode. It should be noticed that even in the full resolution mode, the proposed system achieves superiority in both accuracy and speed over CASSI.
4. Experiments on real captured data
4.1 System implementation
System components. Figure 4(a) demonstrates the prototype system we have built. We use the same objective lenses (Nativar EF $50$ mm, $f/0.95$) to collect and collimate the light, between which we insert a field stop (Thorlabs, CP20S) to limit the field of view. As shown in Fig. 4(b), the notch filter array has seven holes, for which we assemble six notch filters to capture multispectral images and leave one empty to capture the panchromatic image. The center wavelengths of these notch filters are $488nm$, $532nm$, $561nm$, $594nm$, $642nm$, and $785nm$ (Chroma ZET TopNotch). Figure 2(a) shows the transmittance curves of the notch filters. The lens array is composed of seven off-the-shelf plano-convex lenses with a focal length 40mm. The monochromatic sensor (Adimec, S-25A30) with $5120 \times 5120$ pixels, collects six bandstop multispectral images and one panchromatic image in a single snapshot, each image with around $1600 \times 1600$ pixels.
Fig. 4. An image of the prototype system and a schematic diagram of the notch filter array. The center wavelengths (nm) of these notch filters are labeled.
System calibration. The proposed system adopts spatial multiplexing by utilizing a lens array to divide the optical path, so it needs a one-time calibration before the subsequent processing. Specifically, we use a grayscale picture as the target. After the camera exposures, we crop out the images for each lens and pick out a source patch in the panchromatic image. Then the destination patches in other images are searched, as illustrated in Fig. 5. The correlation coefficients between source and destination patches are calculated by the angle of vectors as it can make up the intensity ambiguity,
where $a$ and $b$ denote the vectors of the source patch and destination patch, respectively. $|\cdot |$ and $arccos$ are the modulus and arc cosine operators. As such, the correspondence between the source patch from the panchromatic image and the destination patch from the multispectral images can be recorded as the system parameters.4.2 Experiment results
We first testify our system on multispectral imaging. Figure 6 shows the comparisons between bandpass multispectral images and our bandstop results of scenes Colorchecker and Building. The exposure time for the bandstop filter-based system is set to 40ms (25fps). The exposure time for the bandpass filter-based system is set to 400ms (2.5fps), since it would be totally noise for this system with a 40ms exposure. This phenomenon primarily drives us to develop a high light efficiency system. A colorful reference image is also displayed for each scene to provide context information. It can be seen that the bandpass filter-based system suffers from serious noise even with a relatively low frame rate. In contrast, our system generates the most visually pleasant results with clearer details in the video rate.
Fig. 6. Multispectral image comparisons between bandpass systems and the proposed system for scenes Colorchecker and Building at bands 488nm, 532nm, 561nm, 594nm, 642nm, 785nm. Zoomed regions from three selected bands are displayed, respectively. A colorful reference image is also provided for each scene.
Multispectral video presentation of a human face indoor is also provided in Visualization 1. The exposure time for the bandstop filter-based system is set to 40ms (25fps) and that for the bandpass filter-based system is set to 1000ms (1fps) due to the low light efficiency and the limited light condition. Even with a lower rate, the result images of bandpass filter system still suffer from serious blur, while results of our system look pleasant visually in a faster rate. The acquiring speed superiority of the proposed method has been proved further according to the video.
We then evaluate our system on hyperspectral imaging with spectral super-resolution. Figure 7 shows the spectral super-resolution results for images Colorchecker and Building at five new bands that differ from the stopband of the notch filters, i.e., the images in Fig. 7 can only be obtained with spectral super-resolution. We can see that the HSIs obtained from our system are much clearer than those from bandpass filter-based system. This experiment demonstrates the effectiveness of our system on spectral super-resolution.
Fig. 7. Hyperspectral image comparisons between bandpass systems and the proposed system for scenes Colorchecker and Building at bands 540nm, 580nm, 610nm, 630nm and 650nm. Zoomed regions for band 650nm are displayed. Note the selected bands are difference with the center of the filters thus can only be obtained by spectral super-resolution.
5. Conclusion and discussion
In this work, we propose a novel snapshot spectral imaging system via spatial multiplexing and spectral mixing. The two distinct characteristics, i.e., spatial multiplexing and spectral mixing, guarantee the advantages of low computational burden and high light efficiency, respectively. We have built a prototype system to demonstrate the advantages of our system. Moreover, the proposed system is friendly with spectral super-resolution. We have conducted both simulations on synthetic data, and experiments on real captured data to verify the effectiveness of our systems. Our system is quite flexible as it can easily adjust the number and type of the notch filters to fit specific demands in the real applications.
Our system still has some limitations. First, it must face the tradeoff between spatial, spectral and temporal resolutions, which limits the system more suitable for specific applications where the presence of the temporal information takes precedence over high spatial and spectral resolutions. Second, due to the limited depth of field of the system, the measurements of scenes with great depth difference will introduce depth error [32,33] by the lens array. We utilize a zoom lens with short focal length and small aperture to expand the depth of field of the system. Therefore we can approximate that the sub images are spatially consistent to simplify the problem and reduce the computational complexity. Further works will go from two paths. On the one hand, we will delve into the spatial-spectral resolution tradeoff. Our system could be upgraded for capturing a high-resolution panchromatic image. Thus we can use the pansharpening technique [34] to promote the spatial resolution of the spectral images. On the other hand, we would take advantage of the depth information to structure a high framerate spatial-volumetric-spectral 4D imaging system.
Funding
National Natural Science Foundation of China (61701025, 61922014); Beijing Municipal Science and Technology Commission (Z181100003018003); Beijing Institute of Technology Research Fund Program for Young Scholars.
Disclosures
The authors declare that there are no conflicts of interest related to this article.
References
1. J. James, Spectrograph design fundamentals (Cambridge University, 2007).
2. A. F. H. Goetz, G. Vane, J. E. Solomon, and B. N. Rock, “Imaging spectrometry for earth remote sensing,” Science 228(4704), 1147–1153 (1985). [CrossRef]
3. W. Cho, J. Jang, A. Koschan, M. A. Abidi, and J. Paik, “Hyperspectral face recognition using improved inter-channel alignment based on qualitative prediction models,” Opt. Express 24(24), 27637–27662 (2016). [CrossRef]
4. R. T. Kester, N. Bedard, G. Liang, and T. S. Tkaczyk, “Real-time snapshot hyperspectral imaging endoscope,” J. Biomed. Opt. 16(5), 056005 (2011). [CrossRef]
5. L. Gao and L. V. Wang, “A review of snapshot multidimensional optical imaging: Measuring photon tags in parallel,” Phys. Rep. 616, 1–37 (2016). [CrossRef]
6. N. A. Hagen and M. W. Kudenov, “Review of snapshot spectral imaging technologies,” Opt. Eng. 52(9), 090901 (2013). [CrossRef]
7. G. Arce, D. Brady, L. Carin, H. Arguello, and D. Kittle, “Compressive coded aperture spectral imaging: An introduction,” IEEE Signal Process. Mag. 31(1), 105–115 (2014). [CrossRef]
8. X. Cao, T. Yue, X. Lin, S. Lin, X. Yuan, Q. Dai, L. Carin, and D. J. Brady, “Computational snapshot multispectral cameras: toward dynamic capture of the spectral world,” IEEE Signal Process. Mag. 33(5), 95–108 (2016). [CrossRef]
9. Z. Xiong, L. Wang, H. Li, D. Liu, and F. Wu, “Snapshot hyperspectral light field imaging,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (IEEE, 2017), pp. 3270–3278.
10. L. Wang, Z. Xiong, D. Gao, G. Shi, and W. Feng, “High-speed hyperspectral video acquisition with a dual-camera architecture,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (IEEE, 2015), pp. 4942–4950.
11. L. Wang, Z. Xiong, G. Shi, W. Zeng, and F. Wu, “Simultaneous depth and spectral imaging with a cross-modal stereo system,” IEEE Trans. Circuits Syst. Video Technol. 28(3), 812–817 (2018). [CrossRef]
12. X. Yuan, T. H. Tsai, R. Zhu, P. Llull, D. Brady, and L. Carin, “Compressive hyperspectral imaging with side information,” IEEE J. Sel. Top. Signal Process. 9(6), 964–976 (2015). [CrossRef]
13. M. Descour and E. Dereniak, “Computed-tomography imaging spectrometer: experimental calibration and reconstruction results,” Appl. Opt. 34(22), 4817–4826 (1995). [CrossRef]
14. A. Wagadarikar, R. John, R. Willett, and D. Brady, “Single disperser design for coded aperture snapshot spectral imaging,” Appl. Opt. 47(10), B44–B51 (2008). [CrossRef]
15. X. Lin, Y. Liu, J. Wu, and Q. Dai, “Spatial-spectral encoded compressive hyperspectral imaging,” ACM Trans. Graph. 33(6), 1–11 (2014). [CrossRef]
16. R. Shogenji, Y. Kitamura, K. Yamada, S. Miyatake, and J. Tanida, “Multispectral imaging using compact compound optics,” Opt. Express 12(8), 1643–1655 (2004). [CrossRef]
17. S. A. Mathews, “Design and fabrication of a low-cost, multispectral imaging system,” Appl. Opt. 47(28), F71–F76 (2008). [CrossRef]
18. A. Gorman, D. W. Fletcher-Holmes, and A. R. Harvey, “Generalization of the lyot filter and its application to snapshot spectral imaging,” Opt. Express 18(6), 5602–5608 (2010). [CrossRef]
19. H. Du, X. Tong, X. Cao, and S. Lin, “A prism-based system for multispectral video acquisition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (IEEE, 2009), pp. 175–182.
20. X. Cao, H. Du, X. Tong, Q. Dai, and S. Lin, “A prism-mask system for multispectral video acquisition,” IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2423–2435 (2011). [CrossRef]
21. T. Mu, F. Han, D. Bao, C. Zhang, and R. Liang, “Compact snapshot optically replicating and remapping imaging spectrometer (ORRIS) using a focal plane continuous variable filter,” Opt. Lett. 44(5), 1281–1284 (2019). [CrossRef]
22. M. Gehm, R. John, D. Brady, R. Willett, and T. Schulz, “Single-shot compressive spectral imaging with a dual-disperser architecture,” Opt. Express 15(21), 14013–14027 (2007). [CrossRef]
23. L. Wang, T. Zhang, Y. Fu, and H. Huang, “Hyperreconnet: Joint coded aperture optimization and image reconstruction for compressive hyperspectral imaging,” IEEE Trans. on Image Process. 28(5), 2257–2270 (2019). [CrossRef]
24. S. Zhang, L. Wang, Y. Fu, X. Zhong, and H. Huang, “Computational hyperspectral imaging based on dimension-discriminative low-rank tensor recovery,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (IEEE, 2019), pp. 10183–10192.
25. L. Wang, Z. Xiong, G. Shi, F. Wu, and W. Zeng, “Adaptive nonlocal sparse representation for dual-camera compressive hyperspectral imaging,” IEEE Trans. Pattern Anal. Mach. Intell. 39(10), 2104–2111 (2017). [CrossRef]
26. G. Asrar, R. Weiser, D. Johnson, E. Kanemasu, and J. Killeen, “Distinguishing among tallgrass prairie cover types from measurements of multispectral reflectance,” Remote Sens. Environ. 19(2), 159–169 (1986). [CrossRef]
27. A. Chakrabarti and T. Zickler, “Statistics of real-world hyperspectral images,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2011), pp. 193–200.
28. J. P. Parkkinen, J. Hallikainen, and T. Jaaskelainen, “Characteristic spectra of munsell colors,” J. Opt. Soc. Am. A 6(2), 318–322 (1989). [CrossRef]
29. F. Yasuma, T. Mitsunaga, D. Iso, and S. K. Nayar, “Generalized assorted pixel camera: postcapture control of resolution, dynamic range, and spectrum,” IEEE Trans. on Image Process. 19(9), 2241–2253 (2010). [CrossRef]
30. B. Arad and O. Ben-Shahar, “Sparse recovery of hyperspectral signal from natural rgb images,” in European Conference on Computer Vision, (Springer, 2016), pp. 19–34.
31. J. M. Bioucas-Dias and M. A. Figueiredo, “A new twist: Two-step iterative shrinkage/thresholding algorithms for image restoration,” IEEE Trans. on Image Process. 16(12), 2992–3004 (2007). [CrossRef]
32. C. Guo, W. Liu, X. Hua, H. Li, and S. Jia, “Fourier light-field microscopy,” Opt. Express 27(18), 25573 (2019). [CrossRef]
33. L. E. Weiss, Y. S. Ezra, S. Goldberg, B. Ferdman, O. Adir, A. Schroeder, O. Alalouf, and Y. Shechtman, “Three-dimensional localization microscopy in live flowing cells,” Nature Nanotechnology pp. 1–7 (2020).
34. V. P. Shah, N. H. Younan, and R. L. King, “An efficient pan-sharpening method via a combined adaptive pca approach and contourlets,” IEEE Trans. Geosci. Remote Sens. 46(5), 1323–1335 (2008). [CrossRef]
