Hyperspectral polarimetric imaging is a technique that simultaneously captures spectral information and polarization state for each pixel [1]. Its application researches have been reported on detecting cancer cells [2] and observing the ocean surface [3].

Various hyperspectral polarimeters, for example, hyperspectral polarimeters using a hyperspectral camera with a linear polarizer [4,5], a hyperspectral $(S_1,S_2)$ polarimeter (Here, $S_i$ ($i\,=\,0-3$) represent Stokes parameters [6].) employing a combination of optically replicating imaging spectrometer and cascaded polarimetric modulation elements [7], and hyperspectral full-Stokes polarimeters with polarization gratings (PGs) by using the computed tomography technique [8,9], have been developed so far. Many of them are, however, hyperspectral polarimeters in the visible region, and there are a few reports in the near-infrared (NIR) region. Compared to visible light, NIR light has the characteristics of being safe to the eye, transparent to biological tissues, and able to make measurements independent of the color of objects. Moreover, analyzing both the polarimetric characteristics and absorption spectrum in the NIR region, where abundant absorption lines specific to organic molecules exist, can offer a synthetic object recognition. Therefore, it is important to expand the variation of the hyperspectral polarimeter for the NIR region and to explore its utilization.

In this Letter, we propose a method of acquiring NIR hyperspectral $S_3$ images by attaching a PG to an NIR hyperspectral imager. Furthermore, as a demonstration of the NIR hyperspectral polarimeter, we performed machine learning to classify plastic plates. Through this demonstration, we show that the NIR hyperspectral $S_3$ imaging has the potential to improve the discrimination precision compared to the conventional NIR hyperspectral imaging.

The hyperspectral $S_3$ imager is realized using a key device, an orthogonal circular type PG (OCPG) [10]. This PG is made of a polymer liquid crystal, and the liquid crystal director is rotated with $(\cos (\pi x/\Lambda ),-\sin (\pi x/\Lambda ))$ in the $x$ axis direction [Fig. 1(a)]. Here, $\Lambda$ represents the grating period. When a phase difference of $\pi$ due to birefringence occurs for the PG design wavelength $\lambda _\mathrm {design}$, the PG gives a plane wave phase of $2\pi x/\Lambda$ for left circularly polarized (LCP) light and a plane wave phase of $-2\pi x/\Lambda$ for right circularly polarized (RCP) light as the Pancharatnam–Berry phase [11–13], resulting in diffraction with 100% efficiency. Therefore, this PG has the function of separating LCP and RCP components into $\pm 1$st orders, respectively [Fig. 1(b)]. The circular polarization separation function is maintained even at wavelengths other than the designed wavelength, although the circular polarization diffraction efficiency decreases to $\eta _\mathrm {1}+\eta _\mathrm {-1}=\sin ^2(\pi \Delta nd/\lambda )$ and zeroth-order light is produced. Here, $\eta _{i}$, $\Delta n$, $d$, and $\lambda$ give the diffraction efficiency of the $i$th order representing the grating period, the birefringence of the PG, the thickness of the PG, and the wavelength of the incident light, respectively.

 

Fig. 1. (a) Crossed-nicol polarization microscope image of the OCPG and its director distribution. (b) OCPG’s diffraction dependence on circular polarization chirality. (c) Our proposed NIR hyperspectral $S_3$ imager. O, objects; PG, orthogonal circular type PG; HC, hyperspectral imager. (d) Experimental setup. LS, halogen light source; P, polarizer; QWP, achromatic quarter-wave plate; S, a sample. (e) Circular polarization diffraction efficiency of the OCPG and the reference intensity spectra $I_\mathrm {ref}$ including the optical characteristics of the hyperspectral imager.

Download Full Size | PDF

This hyperspectral $S_3$ imager consists of a commercial push-broom hyperspectral imager (Eba Japan, SIS-I) and the PG [Fig. 1(c)]. The hyperspectral data cube captured by the hyperspectral $S_3$ imager is spatially divided into the RCP intensity distribution ($I_\mathrm {RCP}(x,y;\lambda )$) and LCP intensity distribution ($I_\mathrm {LCP}(x,y;\lambda )$) of the subject. Since the value of $S_3$ is defined by $S_3=(I_\mathrm {RCP}-I_\mathrm {LCP})/(I_\mathrm {RCP}+I_\mathrm {LCP})$, the hyperspectral $S_3$ image can be reconstructed through image processing. The registration of RCP and LCP intensity distributions was adjusted using a rectangular reference image.

The demonstration experiment system is shown in Fig. 1(d). Light from a halogen light source was converted to RCP light by a polarizer and a quarter-wave plate, and a sample was illuminated with an angle of 30$^\circ$. The samples were three kinds of plastic plates (2 mm thickness): polymethylmethacrylate (PMMA), polycarbonate (PC), and polyvinyl chloride (PVC), respectively. The samples were held on an aluminum plate so that only the 12 mm $\times$ 45 mm window was exposed by a NIR absorbing tape. The hyperspectral $S_3$ imager was placed in front of the sample to capture the scattered light.

Figure 1(e) shows the circular polarization diffraction efficiency of an orthogonal circular type PG and the normalized spectra of the illuminated light taken by the hyperspectral imager alone. The diffraction efficiency of the circularly polarized light agrees well with the theoretical curve shown by the dashed line ($\Delta nd\,=\,0.76$) and reaches a maximum value of 99.9% at 1550 nm. The normalized spectra is the spectral characteristics of the halogen lamp and the optical elements excluding the OCPG. From the circular polarization diffraction efficiencies and the normalized spectra, we adopted the wavelength range from 1100 nm to 1600 nm as the hyperspectral $S_3$ image data.

The results of the hyperspectral $S_3$ imaging are shown in Fig. 2. The hyperspectral $S_3$ data cube had spectral data for every 10 nm, but here we show the $S_3$ distribution for every 50 nm. The image size was $12\times 52$ pixels. PC shows a wavelength-dependent $S_3$ distribution, while PVC and PMMA do not show wavelength dependence of $S_3$. In order to compare the hyperspectral $S_3$ imaging with the conventional hyperspectral imaging, we took a reference hyperspectral $S_3$ image in the absence of samples. Here, defining the RCP and the LCP components of this reference hyperspectral $S_3$ data cube as $I_\mathrm {RCP}^\mathrm {(ref)}(x,y;\lambda )$ and $I_\mathrm {LCP}^\mathrm {(ref)}(x,y;\lambda )$, respectively, we introduce a hyperspectral image of scattered light normalized by the reference hyperspectral image $T(x,y;\lambda )=I(x,y;\lambda )/I^\mathrm {ref}(x,y;\lambda ) = (I_\mathrm {RCP}+I_\mathrm {LCP})/(I_\mathrm {RCP}^\mathrm {(ref)}+I_\mathrm {LCP}^\mathrm {(ref)})$ (Fig. 3). In Fig. 3, it is easy to distinguish between PC and PVC, and PC and PMMA, but it is difficult to distinguish between PVC and PMMA because of their similar hyperspectral images. On the other hand, hyperspectral $S_3$ images can easily distinguish the three types of plastics. This is because the hyperspectral images observe the spatial distribution of the transmission spectrum, whereas the hyperspectral $S_3$ images observe the spatial distribution of polarization characteristics which are given by the $4\times 4$ Muller matrix $\mathbf {M}=\{m_{ij}\}$. In our experiments, the $m_{41}$ and $m_{44}$ components, which respectively correspond to circular dichroism, and birefringence and diattenuation [14], can be measured since the illumination light is $S_3=1$. Since circular dichroism and diattenuation were not observed in the plastic material, we mainly measured birefringence; PVC and PMMA have similar absorption spectral characteristics in the NIR region [15], but the magnitude of birefringence is different, which allows us to clearly distinguish them in the hyperspectral $S_3$ images. In addition, observing $S_3$ images, unlike linearly polarized ($S_1$ or $S_2$) images, has the advantage of being able to detect birefringence even if the orientation of birefringence is unknown.

 

Fig. 2. Hyperspectral $S_3$ images of the scattering light from 1100 nm to 1600 nm.

Download Full Size | PDF

 

Fig. 3. Hyperspectral images of the scattering light normalized by the reference hyperspectral image ($I(\lambda )/I_\mathrm {ref}(\lambda )$).

Download Full Size | PDF

For further quantitative comparison, the measured data were analyzed by principal component analysis (PCA) [16], a machine learning algorithm that re-selects an orthogonal basis. The principal orthogonal vectors selected by the PCA are called the principal components, and its $j$th vector is described by $\left |\mathrm {PC}j\right\rangle$. Then the $i$th measured data vector $\left |\Psi _i\right\rangle$ is written in $\left | \Psi _i \right\rangle = \sum _{j=1}^{N_\mathrm {PCA}} c_{ij}\left | \mathrm {PC}j\right\rangle$, in which $N_\mathrm {PCA}$ is the dimension of data vectors and $c_{ij}$ represents an expansion coefficient. The PCA algorithm that first chooses $\left |\mathrm {PC}1\right\rangle$ so that the variance $V(c_{i1})=N^{-1}\sum _i (c_{i1}-\bar c_{i1})^2$ is maximized, where $N$ is the total number of data vectors and $\bar c_{i1}$ is the average of $c_{i1}$. Then, for the complementary space, $\left |\mathrm {PC}2\right\rangle$ is chosen so that the variance $V(c_{i2})$ is maximized, and so on until $\left |\mathrm {PC}N_\mathrm {PCA}\right\rangle$. This allows higher dimensional vectors to be represented by a linear combination of fewer principal components. This technique is used in dimensionality compression.

Here, the spectral data vectors for hyperspectral $S_3$ data cube and normalized hyperspectral data cube are described by

The principal components (PC1 and PC2) were obtained by performing PCA on the $S_3$ spectral data vectors for three kinds of plastic samples $\left \{\left |S_{3,0}^\mathrm {PC}\right\rangle,\ldots,\left |S_{3,N_\mathrm {d}}^\mathrm {PC}\right\rangle \right.$, $\left |S_{3,0}^\mathrm {PVC}\right\rangle ,\ldots,\left |S_{3,N_\mathrm {d}}^\mathrm {PVC}\right\rangle$, $\left.\left |S_{3,0}^\mathrm {PMMA}\right\rangle,\ldots,\left |S_{3,N_\mathrm {d}}^\mathrm {PMMA}\right\rangle \right \}$ and the normalized hyperspectral data vectors $\left \{\left |T_0^\mathrm {PC}\right\rangle,\ldots,\left |T_{N_\mathrm {d}}^\mathrm {PC}\right\rangle\right.$, $\left |T_0^\mathrm {PVC}\right\rangle,\ldots,\left |T_{N_\mathrm {d}}^\mathrm {PVC}\right\rangle$, $\left |T_0^\mathrm {PMMA}\right\rangle,\ldots$, $\left |T_{N_\mathrm {d}}^\mathrm {PMMA}\right\rangle\}$, where $N_\mathrm {d}$ gives the total number of data vectors for each sample. The $S_3$ spectral data vectors have a smooth spectral dependence for both PC1 and PC2 [Fig. 4(a), blue lines]. This mainly reflects the wavelength dependence of PC birefringence, as seen in Fig. 2. On the other hand, the principal components of the normalized spectral data vectors show complex waveforms [Fig. 4(b), green lines]. Figure 4(b) shows the cumulative explained variance ratio, which is an indicator of the extent to which the original data can be reproduced. While the normalized spectral data vectors require many principal components, the $S_3$ spectral data vectors need only two principal components to achieve a cumulative explained variance ratio of 99.6%. Figures 4(c) and 4(d), respectively, give projection plots of the $S_3$ spectral data vectors and the normalized hyperspectral data vectors as the coefficients of the principal components up to the second. In Fig. 4(c), the spectral data of the three plastics are plotted separately as clusters. This indicates that it is easy to discriminate substances using the discriminant function algorithm. On the other hand, in Fig. 4(d), there are overlaps of the three plastics, leading to misclassification. These results also indicate that the hyperspectral $S_3$ imaging and PCA succeed in object identification.

 

Fig. 4. Results for principal component analysis to analyze characteristics of the spectral data acquired in the experiment. (a) Waveforms of the first and second principal components for $S_3$ spectral data vectors and normalized spectral data vectors ($I/I_\mathrm {ref}$). (b) Cumulative explained variance ratio for $S_3$ spectral data vectors and normalized spectral data vectors. (c) and (d) Projection plots of (c) $S_3$ spectral data vectors and (d) normalized spectral data vectors.

Download Full Size | PDF

Next, we performed an object classification demonstration by using PCA and a discriminant function algorithm. The spectral data in the upper half of a hyperspectral image were used as training data ($N_\mathrm {d}\!=\!300$), and the spectral data in the lower half of hyperspectral images were used as test data ($N_\mathrm {d}\!=\!300$) for machine learning. They contained 51 wavelength data every 10 nm from 1100 nm to 1600 nm. We note that the spectral data per pixel in a single hyperspectral image were used as training and validation data in the context of comparing the performance of conventional hyperspectral imaging and hyperspectral $S_3$ imaging. We used Python (3.11.3) as the machine learning runtime environment and the scikit-learn (1.3.0) library. Voronoi boundaries were determined using LinearSVC [16] as a discriminant function algorithm, with the coefficients of the first and second principal components. The results of substance identification of the hyperspectral $S_3$ data cube are shown in Fig. 5(a). The gray lines are the Voronoi boundaries determined by the training data, and the plotted points indicate the principal component coefficients of the test data. Figure 5(c) shows the discrimination results for the test data on the $(x,y)$ plane. The accuracy achieved is 100%. Figures 5(b) and 5(d) show the results of the same pipeline processing on the normalized hyperspectral data cube. The accuracy of classification was 95.0%, since PVC and PMMA were misclassified as PC at the edges of the image.

 

Fig. 5. Classification demonstration results. (a) and (b) Projection plots of (a) $S_3$ spectral data vectors and (b) normalized spectral data vectors on the principal component space with Voronoi boundaries (gray lines). (c) and (d) Classification results of the test data [the lower half of (c) hyperspectral $S_3$ images and (d) normalized hyperspectral images] on the $(x,y)$ plane.

Download Full Size | PDF

We examined the classification performance of this machine learning using the cross-validation method [16], in which the training and test data are divided randomly. The number of data partitions for the cross-validation method was set to 5. The accuracies of the hyperspectral $S_3$ data cube and the normalized hyperspectral data cube were $100{\% }$ and $96.3\pm 0.6{\% }$, respectively. From these results, it was found that the performance of the hyperspectral $S_3$ data cube was better than that of the normalized hyperspectral data cube, regardless of the choice of training data.

In this demonstration, the hyperspectral $S_3$ image showed better performance in object classification than the normalized hyperspectral image. This is because the samples had similar absorption spectrum but had different wavelength characteristics of birefringence. Furthermore, since the wavelength dispersion of the birefringence was smoother than that of the absorption spectrum, the principal components extracted by PCA were noise-resistant.

In the material classification demonstration of the normalized hyperspectral image, a significant misclassification occurred at the edges of the image [Fig. 5(d)]. This is because the normalized hyperspectral images were affected by the reference hyperspectral image acquired separately. By comparing Fig. 2 with Fig. 3, the normalized hyperspectral images remarkably changed its value at the edge of the images, but the change at the edge of hyperspectral $S_3$ images was insignificant. Hyperspectral $S_3$ imaging does not require any reference images and is normalized by its own intensity, so it is less likely to have edge problems and also less likely to produce uneven images (Fig. 2).

We observed the birefringence of plastic samples through hyperspectral $S_3$ imaging. The key point of this experiment is that the samples were illuminated with the circularly polarized light. In the case of unpolarized illumination, only $m_{41}$ of the Muller matrix related to circular dichroism can be measured, and birefringence cannot be detected. In the case of linearly polarized illumination, it is difficult to detect birefringence if the optic axes of the sample’s birefringence and the orientation of the linearly polarized light are close. The light source selection is thus important in the acquisition of polarization characteristics.

Recent studies have reported polarimetric imaging using metasurface PGs [17,18], which can be combined with a hyperspectral camera to construct a full-Stokes hyperspectral polarimeter as a future work.

In this study, we attached a PG to a commercial hyperspectral camera to acquire NIR hyperspectral $S_3$ images of plastic plates. We then used machine learning to classify the three types of plastic plates. Through this demonstration, the NIR hyperspectral $S_3$ images showed higher classification precision than the conventional normalized NIR hyperspectral images. This result indicates that the NIR hyperspectral $S_3$ imaging may facilitate object classification even for samples with similar absorption spectra. This hyperspectral camera can be applied such as in garbage classification in recycling plants.