Open Access
Add to BibSonomy
Abstract
Commercially available hyperspectral cameras are useful for remote sensing, but in most cases snapshot imaging is difficult due to the need for scanning. The coded aperture snapshot spectral imager (CASSI) has been proposed to simultaneously acquire a target scene’s spatial and spectral dimensional data, employing a refractive prism as a disperser. This paper proposes a CASSI-based technique using a generally available diffraction grating of a Ronchi ruling and blazed grating and its improvement using the undispersed zeroth-order light. The feasibility and performance of the proposed technique are experimentally validated, and the grating parameters are identified.
© 2022 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. INTRODUCTION
Hyperspectral (HS) imaging is essential acquiring the HS data cube of an analyzed scene, which contains multiple two-dimensional (2D) spectroscopic intensity distributions in a variety of fields, such as remote sensing [1,2], bioimaging and medical diagnosis [3,4], and product inspection for materials, and foods [5,6]. Such three-dimensional (3D) data cannot be acquired in a snapshot manner without lack of information on either condition with a detector of a linear sensor or an array sensor due to compression to lower-dimensional data in a single frame. Several approaches can achieve complete spectral image acquisition, such as laterally scanning the sensors [7,8], sweeping a detected wavelength using multiple bandpass filters or a tunable filter [9], and spatially dividing imager pixels to detect corresponding wavelengths with pixel-wise bandpass filters [10]. However, these approaches can diminish the time or spatial resolution. In this paper, we focused on the technique based on compressive sensing (CS) though single-shot techniques based on the speckle correlation [11]. CS is an effective solution for the ill-posed problem, such as this case to retrieve the HS data cube from a 2D intensity distribution. Some CS-based single-shot spectral imaging techniques have been proposed [12–16], such as the coded aperture snapshot spectral imager (CASSI) [17,18].
The CASSI system consists of randomly arranged binary apertures, coded apertures, dispersive elements such as a refractive prism, and a monochromatic image sensor. An analyzed target scene is encoded with the coded aperture and projected onto the image sensor plane with lateral shifts (ordinary along the $x$ direction) according to the spectral components by prism dispersion. The shifted intensity distributions and their corresponding wavelengths’ overlap were analyzed through a CS-based reconstruction algorithm. Then, the respective spectral images, or HS data cube, are estimated. CASSI is used to acquire the HS data cube using generally available optical components with only a single measurement, which has recently been applied to various applications, e.g., spectral imaging of an object behind a thin scatterer [19] and ultrafast imaging using a streak camera [20].
Although CASSI usually employs a refractive prism as a disperser due to its high light-use efficiency, it restricts the performance and setup size of the CASSI system. In essence, the spectral resolution of a measured HS data cube depends on the pixel pitch of the image sensor and the magnitude of the dispersion. In other words, the narrower the wavelength width that enters a pixel column perpendicular to the lateral shift direction, the higher the spectral resolution of the HS data cube. Increasing the lateral shift for respective dispersed spectral components effectively improves the spectral resolution of a CASSI system because of the finite-sized imager pixel. However, a prism with increased dispersion results in a bulky CASSI system due to the need for a larger one farther from an imager, since the prism is made from a fixed material such as glass. There is also the problem of low spatial resolution in measured spectral images caused by a lack of information associated with a high compression ratio in the 2D detection of 3D data cubes. To solve this problem, Kittle et al. proposed a multiframe scheme with variable coded apertures at the group which presented the original CASSI paper [21], which provides better image quality. In contrast, the scheme can spoil the advantage of the single-shot potential of CASSI due to time-resolved measurements. The dual-camera scheme has been reported by Wang et al. [22] as another solution. This method uses an uncoded and undispersed image captured by a general imaging system capturing through CASSI, which improves the spatial resolution due to the high-frequency structure (edge parts) contained in the image. Although the single-shot measurement is possible with better reconstruction quality, another optical path and extra elements for the additional imaging system are required, making the whole system complicated.
To address the aforementioned problems, we utilize a commercially available diffraction grating with either a Ronchi ruling (with periodically arranged binary slits) or a blazed grating (with sawtooth groove profile) as a disperser for CASSI. Utilizing a diffraction grating can decreases the light-use efficiency. Unlike a prism, it includes components that are not always necessary for the HS reconstruction for zeroth-order light without dispersion and higher- or negative-order light. We have reported the compressive spectral imaging scheme using a blazed grating [23]. We employ both first- and zeroth-order diffracted light, which increases light-use efficiency. Furthermore, encoded but not dispersed, the zeroth-order light contains more high-frequency components than first-order light, similar to the dual-camera scheme. These properties improve CS-based estimation and ensure high-quality spectral images are acquired. Recently, a similar scheme has been reported [24] that also achieves good image reconstruction, but did not refer to selection or determination of a grating parameter. Accordingly, we present a compressive spectral imaging technique using a Ronchi ruling and a blazed grating with their detailed parameter determination in the present paper. These conditions are compared in optical experiments, and their capabilities are discussed.
2. OVERVIEW OF COMPRESSIVE SPECTRAL IMAGING USING A DIFFRACTION GRATING
A. Optics and Reconstruction Framework
In basic CASSI, information lacks the target scene, particularly in the figuration (e.g., the profile) because lateral shifts of coded intensity distributions overlap with their corresponding wavelengths. This is supplemented by estimation, which results in low-resolution spectral images. The proposed method is inspired by the dual-camera scheme [22] and uses undispersed zeroth-order light for mitigating the decrease of the spatial resolution.
Figure 1 shows the optical setup for the proposed method, which is composed of imaging lenses, a coded aperture, and a monochromatic image sensor similar to a basic CASSI system except for employing a diffraction grating as a disperser. Under $x$-direction shift by dispersion, a target scene is expressed as the power spectral density $f(x,y,\lambda)$, and the intensity distributions of the first- and zeroth-order lights detected by the image sensor are, respectively, described as
Spectral information is extracted from dispersed first-order light. Higher spatially resolved figuration-related information is extracted from dispersionless zeroth-order light through the above framework in the proposed method. Measurements with in-line optics can be achieved because first- and zeroth-order light is detectable in the common imager plane from the property of a diffraction grating though an area on an imager to capture a scene is reduced.
B. Property for the Type of Diffraction Grating
The proposed method supposes the use of a Ronchi ruling or a blazed grating as a disperser. Unlike a refractive prism, the gratings generate dispersed first-, zeroth-, negative-, and higher-order light. In the case of a Ronchi ruling, the diffraction efficiency is independent of a target-scene wavelength. The ratio of a first-order diffracted light to a zeroth-order one is theoretically fixed. On the other hand, in the case of a blazed grating, the diffraction efficiency depends on both the target-scene wavelength and the blaze wavelength. In particular, the latter is critical in determining the performance of the system.
A blazed grating is designed to ensure maximum diffraction efficiency at the specific order (ordinarily first order); it concentrates incident light energy to the first order at the blaze wavelength and decreases the zeroth order and others. Although the blaze wavelength is usually set to maximize the diffraction efficiency of a specific target-scene wavelength, the mismatch between the blaze wavelength and target spectral band would rather work well for HS imaging. As the proposed method utilizes both first- and zeroth-order light, the signal-to-noise ratio (SNR) of distributions detected by the image sensor should be balanced.
For example, if the blaze wavelength of the ultraviolet range (300 nm) is employed for a naturally existing scene with a visible range of approximately 400–700 nm, the wavelength dependencies of the diffraction efficiency for first- and zeroth-order light can be shown in Fig. 2 (the raw data of Fig. 2 are released from the vendor of the blazed grating of GTU13-03, which is described later). The diffraction efficiency of first-order light decreases as the wavelength increases, and that of zeroth-order light inversely increases. Most notably, the average intensities of these light orders are comparable in the visible range, improving the SNR in the reconstruction.
Fig. 2. Wavelength dependence of diffraction efficiency in a blazed grating with a blaze wavelength of 300 nm.
Table 1. Experimental Parameters of Each Element
3. EXPERIMENTAL DEMONSTRATION AND DISCUSSION
The reconstruction of HS data cubes by the proposed method was experimentally validated. The experiments were performed under three conditions: first-order diffracted light; first- and zeroth-order diffracted light with a Ronchi ruling (Edmond Optics, 200 lines/mm); and first- and zeroth-order diffracted light with a blazed grating (Thorlabs Inc., GTU13-03, 300 grooves/mm) with a blaze wavelength of 300 nm. The schematic of the experimental setup is shown in Fig. 3. Table 1 shows the experimental parameters. Measured targets are the star- and triangle-shaped objects made from bandpass filters with center wavelengths of 600 and 450 nm and a full width at half-maximum (FWHM) of 50 and 25 nm, respectively. Although a transmission-type coded aperture created by microfabrication may be useful for a linear configuration, the setup has a folded-optics configuration. The aperture is implemented as an element for reflection-type optics by displaying the binary random pattern, shown in Fig. 4, on a digital micromirror device (DMD) display. The modulated light was projected onto an image sensor (Baumer AG, LXG-120) through the diffraction grating, with the first- and zeroth-order images fitting within the image sensor area. A Ronchi ruling and blazed grating were 44 and 23 mm, respectively, apart from the image sensor to ensure that the positions of first- and zeroth-order images projected onto the sensor were constant, as shown in Fig. 5. Here the pixel in the coded aperture was projected across
Fig. 6. Estimated spectral images by (a) first-order light with a Ronchi ruling; (b) first- and zeroth-order light with a Ronchi ruling; and (c) first- and zeroth-order light with a blazed grating. The numbers displayed in the upper right of each image are the corresponding wavelengths.
Spectral images reconstructed by the three conditions and the associated color synthesized images are shown in Figs. 6 and 7, respectively. Under all conditions, the spectral images reflecting the wavelengths contained in the targets seem to be reconstructed, but the edge shape of reconstructed targets is distorted in Figs. 6(a) and 7(a), which suggests that employing only first-order images results in figuration-related information loss due to the superposition of shifted and overlapped spectral images. In contrast, from Figs. 6(b), 6(c) and 7(b), and 7(c), it is evident that the reconstructed targets preserve their spatial features, which implies that the undispersed image enables us to estimate the features accurately. In Fig. 7(a), the estimated colored objects are distorted like the chromatic aberration. This is caused by the distorted coded aperture image of the first-order diffracted light due to various factors such as the setup alignment and the blur by optics. However, the proposed method can correctly measure the spectral information from such a low-quality captured image with the help of the zeroth-order image.
To examine whether the spectral components contained in reconstructed targets were estimated correctly, the results were compared with those measured by a spectrometer (PhotonTec Berlin GmbH, Firefly 4000) as the ground truth. Figure 8 shows the spectral intensities at points P1 and P2 on the reconstructed targets in Fig. 7. For P1 on the star-shaped object with longer wavelengths, both methods roughly estimate the same spectral intensities as those measured by a spectrometer in the range of ${\pm}30\;{\rm{nm}}$ from the center wavelength, but estimation errors occur outside this range. On the other hand, the estimated results for P2 on the triangle-shaped object with shorter wavelengths are different from those of the star-shaped object. The spectral intensities are estimated accurately within the range of ${\pm}20\;{\rm{nm}}$ from the center wavelength with first-order light by a Ronchi ruling. However, with both first- and zeroth-order light by a blazed grating, the results obtained for both first- and zeroth-order light by a Ronchi ruling have some errors, which is evident in the FWHM values. The difference is due to the SNR of first-order images, discussed below. The higher intensity first-order light captured by the method with a blazed grating works well for estimating spectral information. On the other hand, either method incorrectly estimates the higher spectral intensities out of the spectral range of an object than the ground truth. Even for errors out of the range, the method using first- and zeroth-order light with a blazed grating is relatively superior. These results imply that employing zeroth-order light by a grating is effective for estimating spatial information, and a blazed grating with an appropriate blaze wavelength is useful in accurately reconstructing the spectral information. In addition, the edge parts of an object were evaluated for comparison of the capability to estimate figuration-related information. Cross-sectional profiles of the star-shaped object for respective spectral images at 600 nm are shown in Fig. 9. The edges of the estimated image with only the first-order light are gradual, while the ones estimated by both methods with first- and zeroth-order light are steep. These results indicate that the proposed method employing the zeroth-order light in addition to the dispersed first-order light can estimate spectral images with higher spatial resolution.
Fig. 8. Comparison of spectral intensities at (a) P1 with a center wavelength of 600 nm and a FWHM of 50 nm and at (b) P2 with a center wavelength of 450 nm and a FWHM of 25 nm.
As the above results show, appropriately determining the grating parameter of the blaze wavelength is the key factor for the estimation, with better quality in the proposed method using a blazed grating. For a blaze wavelength of 300 nm, the diffraction efficiency of first-order light is approximately twice as large as that of zeroth-order light at 450 nm, and vice versa at 600 nm, as shown in Fig. 2. Therefore, the spectral result of P2 at 450 nm is optimal due to the higher intensity of the dispersed first-order light compared with P1 at 600 nm. In addition, since the spatial feature of the triangle-shaped object is not significantly impaired despite the lower zeroth-order light, it is desirable to determine the blaze wavelength so that the diffraction efficiency of first-order light is larger than that of zeroth-order light. However, it is difficult to equalize the spectral estimation performance over the whole spectral range in a target scene because first- and zeroth-order light have opposing wavelength dependencies with respect to the diffraction efficiency, as mentioned in Section 2. Introducing a wavelength-dependent variable-weighted parameter to regularization in the reconstruction algorithm as $\tau (\lambda)$ can effectively mitigate differences in reconstruction quality due to different wavelength responses. In addition, two narrowband objects were targeted, but since the superposition of first-order light increases when a wideband and multiple objects are targeted, it is necessary to investigate how to determine the blaze wavelength in such a condition in more detail.
4. CONCLUSION
In this paper, we presented a CASSI-based HS imaging technique using a generally available diffraction grating of a Ronchi ruling and blazed grating as a disperser and employed both first- and zeroth-order light for the reconstruction. The proposed method enables its optics to keep small in case the wavelength resolution becomes larger by using a grating with a larger lattice constant and improves the reconstruction quality owing to the undispersed zeroth-order light containing more figuration-related information. The performance was experimentally validated by comparing the following cases: first-order light by a Ronchi ruling; both first- and zeroth-order light by a Ronchi ruling; and both first- and zeroth-order light by a blazed grating. The reconstructed spectral images were distorted on the edges of the objects for first-order light, whereas the other two cases reconstructed spectral images accurately. The spectral fidelity of the reconstructed images by a blazed grating was superior to that by a Ronchi ruling for zeroth-order light. These results show that undispersed zeroth-order light can accurately reconstruct spatial features in a target scene; moreover, a blazed grating provides more accurate spectral reconstruction than a Ronchi ruling. However, the blaze wavelength must be appropriately determined in blazed grating. Since the diffraction efficiency of first-order light influences the spectral reconstruction more than it does figuration-related information, it is desirable to determine the blaze wavelength so that the diffraction efficiency of first-order light is larger than that of zeroth-order light. The proposed method is useful for agriculture and biomedical fields due to its high-quality (spatial and spectral) snapshot measurement of the HS data cube and its simple configuration.
Funding
Japan Society for the Promotion of Science (JP19K20401).
Disclosures
The authors declare no conflicts of interest.
Data Availability
The data that support the findings of this study are available from the corresponding author upon reasonable request.
REFERENCES
1. J. S. Tyo, D. L. Goldstein, D. B. Chenault, and J. A. Shaw, “Review of passive imaging polarimetry for remote sensing applications,” Appl. Opt. 45, 5453–5469 (2006). [CrossRef]
2. Y. Wang, M. E. Pawlowski, S. Cheng, J. G. Dwight, R. I. Stoian, J. Lu, D. Alexander, and T. S. Tkaczyk, “Light-guide snapshot imaging spectrometer for remote sensing applications,” Opt. Express 27, 15701–15725 (2019). [CrossRef]
3. Y. Lv, J. Zhang, D. Zhang, W. Cai, N. Chen, and J. Luo, “In vivo simultaneous multispectral fluorescence imaging with spectral multiplexed volume holographic imaging system,” J. Biomed. Opt. 21, 060502 (2016). [CrossRef]
4. X. Gao, Y. Cui, R. M. Levenson, L. W. K. Chung, and S. Nie, “In vivo cancer targeting and imaging with semiconductor quantum dots,” Nat. Biotechnol. 22, 969–976 (2004). [CrossRef]
5. X. Fu and J. Chen, “A review of hyperspectral imaging for chicken meat safety and quality evaluation: application, hardware, and software,” Compr. Rev. Food Sci. Food Saf. 18, 535–547 (2019). [CrossRef]
6. H. Pu, L. Lin, and D.-W. Sun, “Principles of hyperspectral microscope imaging techniques and their applications in food quality and safety detection: a review,” Compr. Rev. Food Sci. Food Saf. 18, 853–866 (2019). [CrossRef]
7. P. Mouroulis, R. O. Green, and T. G. Chrien, “Design of pushbroom imaging spectrometers for optimum recovery of spectroscopic and spatial information,” Appl. Opt. 39, 2210–2220 (2000). [CrossRef]
8. M. E. Gehm, M. S. Kim, C. Fernandez, and D. J. Brady, “High-throughput, multiplexed pushbroom hyperspectral microscopy,” Opt. Express 16, 11032–11043 (2008). [CrossRef]
9. X. Wang, Y. Zhang, X. Ma, T. Xu, and G. R. Arce, “Compressive spectral imaging system based on liquid crystal tunable filter,” Opt. Express 26, 25226–25243 (2018). [CrossRef]
10. G. Themelis, J. S. Yoo, and V. Ntziachristos, “Multispectral imaging using multiple-bandpass filters,” Opt. Lett. 33, 1023–1025 (2008). [CrossRef]
11. S. K. Sahoo, D. Tang, and C. Dang, “Single-shot multispectral imaging with a monochromatic camera,” Optica 4, 1209–1213 (2017). [CrossRef]
12. O. F. Kar and F. S. Oktem, “Compressive spectral imaging with diffractive lenses,” Opt. Lett. 44, 4582–4585 (2019). [CrossRef]
13. M. A. Golub, A. Averbuch, M. Nathan, V. A. Zheludev, J. Hauser, S. Gurevitch, R. Malinsky, and A. Kagan, “Compressed sensing snapshot spectral imaging by a regular digital camera with an added optical diffuser,” Appl. Opt. 55, 432–443 (2016). [CrossRef]
14. M. Zhang, L. Wang, L. Zhang, and H. Huang, “High light efficiency snapshot spectral imaging via spatial multiplexing and spectral mixing,” Opt. Express 28, 19837–19850 (2020). [CrossRef]
15. P. Wang and R. Menon, “Computational multispectral video imaging,” J. Opt. Soc. Am. A 35, 189–199 (2018). [CrossRef]
16. K. Monakhova, K. Yanny, N. Aggarwal, and L. Waller, “Spectral DiffuserCam: lensless snapshot hyperspectral imaging with a spectral filter array,” Optica 7, 1298–1307 (2020). [CrossRef]
17. A. A. Wagadarikar, R. John, R. Willett, and D. J. Brady, “Single disperser design for coded aperture snapshot spectral imaging,” Appl. Opt. 47, B44–B51 (2008). [CrossRef]
18. A. A. Wagadarikar, N. P. Pitsianis, X. Sun, and D. J. Brady, “Video rate spectral imaging using a coded aperture snapshot spectral imager,” Opt. Express 17, 6368–6388 (2009). [CrossRef]
19. X. Li, J. A. Greenberg, and M. E. Gehm, “Single-shot multispectral imaging through a thin scatterer,” Optica 6, 864–871 (2019). [CrossRef]
20. P. Wang, J. Liang, and L. V. Wang, “Single-shot ultrafast imaging attaining 70 trillion frames per second,” Nat. Commun. 11, 2091 (2020). [CrossRef]
21. D. Kittle, K. Choi, A. Wagadarikar, and D. Brady, “Multiframe image estimation for coded aperture snapshot spectral imagers,” Appl. Opt. 49, 6824–6833 (2010). [CrossRef]
22. L. Wang, Z. Xiong, D. Gao, G. Shi, and F. Wu, “Dual-camera design for coded aperture snapshot spectral imaging,” Appl. Opt. 54, 848–858 (2015). [CrossRef]
23. D. Shimoyama, Y. Saita, and T. Nomura, “Compressive spectral imaging with a blazed grating to improve the signal-to-noise ratio,” in OSJ-OSA-OSK Joint Symposia on Optics Program (2020), paper 14pBJ4.
24. J. Hlubuček, J. Lukeš, J. Václavík, and K. Žídek, “Enhancement of CASSI by a zero-order image employing a single detector,” Appl. Opt. 60, 1463–1469 (2021). [CrossRef]
25. J. Bioucas-Dias and M. Figueiredo, “A new TwIST: two-step iterative shrinkage/thresholding for image restoration,” IEEE Trans. Image Process. 16, 2992–3004 (2007). [CrossRef]
26. P. Getreuer, “Total variation inpainting using split Bregman,” Image Process. Line 2, 147–157 (2012). [CrossRef]
