Deep learning reconstruction enables full-Stokes single compression in polarized hyperspectral imaging

Figure 1. Overall schematic diagram of DL-FSCPHI method

Due to the rich information reflected, polarized hyperspectral imaging has been widely applied in environmental monitoring, biological diagnosis, food safety, and other fields. In terms of technology development, polarized imaging is mainly based on Fourier transform, pixelated polarizers, and compressive sensing (CS). Currently, all the above three methods can achieve full-Stokes polarized imaging. Typically, Fourier transform imaging spectropolarimetry based on polarization modulation array (PMAFTISP) requires only one acquisition to obtain full-Stokes images. The PMAFTISP includes three polarization modulation arrays and three independent optical elements. System complexity and channel crosstalk may affect imaging quality. In addition, pixelated full-Stokes polarimeters require rotating polarizers or designing metasurfaces. Moreover, the fabrication of precision pixelated devices is costly and time-consuming. Recently, compressive full-Stokes polarimeters are constructed with only two commercial components, providing an easy-to-operate and time-saving system. Full-Stokes images can be reconstructed from two measurements compressed by a quarter-wave plate (QWP) and a liquid crystal tunable filter (LCTF). Furthermore, benefiting from a retarder followed by a Wollaston prism with splitting effect, full-Stokes images can be reconstructed from one measurement. Nevertheless, the above compressive polarimeters all rely on traditional reconstruction methods, such as two-step iterative shrinkage/threshold (TwIST) algorithm, which require careful selection of polarization parameters and sparse basis.

The research team of Professor Xu Tingfa from Beijing University of Technology has developed full-Stokes single compression in polarized hyperspectral imaging by introducing deep learning reconstruction (DL-FSCPHI). Full-Stokes images of the target light are compressed into one measurement by a QWP combined with an LCTF. The full Stokes images are then reconstructed simultaneously by a convolutional neural network (CNN). The relevant results were published in Volume 21, Issue 5 of Chinese Optics Letters (Axin Fan, et al. Deep learning reconstruction enables full-Stokes single compression in polarized hyperspectral imaging) and was selected as the cover paper.

The shown figure 1 illustrates the overall schematic diagram of DL-FSCPHI method comprised of imaging system and polarization reconstruction. The imaging system mainly consists of a light source (Thorlabs, OSL2), a QWP (Thorlabs, SAQWP05M-700), an LCTF (Thorlabs, KURIOS-VB1/M), and a complementary metal oxide semiconductor (CMOS) detector (Basler, acA2040-180km). The polarization state of the target light is expressed by a column vector composed of four Stokes parameters. The polarization characteristics of both QWP and LCTF are described by a Mueller matrix with 16 elements in four rows and four columns. Furthermore, LCTF and QWP Mueller matrices are multiplied to obtain the system Mueller matrix. Then, the system Mueller matrix is multiplied by the Stokes column vector of the target light to obtain the Stokes column vector of the modulated light. The modulated first Stokes parameter representing the total light intensity is finally measured by the CMOS detector. Herein, the QWP and LCTF polarization angles are fixed and the LCTF center wavelength is switched to obtain the polarized hyperspectral images of the target under full-Stokes single compression.

The polarization reconstruction involves model training and model testing. Initially, a CNN model composed of two convolution layers is built on Keras framework. The first convolution layer expands one compressed image into multiple images, and the second convolution layer enhances the image details to predict full-Stokes images. The model is then trained utilizing full-Stokes images and compressed images of 60 targets with 400×400 spatial pixels in 18 spectral bands. The model is tested utilizing full-Stokes images and compressed images of other 7 targets with 400×400 spatial pixels in 18 spectral bands. In order to fully verify the reliability of DL-FSCPHI method, two polarization angles, two model structures, and two training parameters are designed. Compared with typical TwIST algorithm, the average peak signal to noise ratio (PSNR) and structural similarity (SSIM) values are improved by 13.55 dB and 0.28, respectively.

This work demonstrates the great promise to develop deep learning reconstruction for full-Stokes single compression and other applications. In the future, it is worth investigating a stronger universal model that is widely applicable to different imaging systems and different polarization angles. Meanwhile, deep learning reconstruction is being extended to four-dimensional compressive imaging, including one-dimensional polarized, one-dimensional spectral, and two-dimensional spatial compression.



图 1 DL-FSCPHI方法总体原理图


北京理工大学许廷发教授研究团队通过引入深度学习重建,开发了全斯托克斯单次压缩的偏振高光谱成像(DL-FSCPHI)系统。目标光的全斯托克斯图像由QWP与LCTF联合压缩到一次测量中,然后通过卷积神经网络(CNN)同时重建完整的斯托克斯图像。相关成果发表在Chinese Optics Letters第21卷第5期上(Axin Fan, et al. Deep learning reconstruction enables full-Stokes single compression in polarized hyperspectral imaging),并被选为当期封面。


偏振重建包括模型训练和模型测试。首先,在Keras框架上构建由两个卷积层组成的CNN模型。第一层卷积层将一幅压缩图像扩展成多幅图像,第二层卷积层增强图像细节以预测全斯托克斯图像。然后,利用60个目标在18个光谱波段下具有400×400个空间像素的全斯托克斯图像和压缩图像训练模型,利用另外7个目标在18个光谱波段下同样具有400×400个空间像素的全斯托克斯图像和压缩图像测试模型。为了充分验证DL-FSCPHI方法的可靠性,设计了两个偏振角度、两种模型结构和两组训练参数。与典型TwIST算法相比,平均峰值信噪比(PSNR)和结构相似度(SSIM)值分别提高了13.55 dB和0.28。