Open Access
Add to BibSonomy
Abstract
Polarimetric imaging can become challenging in degraded environments such as low light illumination conditions or in partial occlusions. In this paper, we propose the denoising convolutional neural network (DnCNN) model with three-dimensional (3D) integral imaging to enhance the reconstructed image quality of polarimetric imaging in degraded environments such as low light and partial occlusions. The DnCNN is trained based on the physical model of the image capture in degraded environments to enhance the visualization of polarimetric imaging where simulated low light polarimetric images are used in the training process. The DnCNN model is experimentally tested on real polarimetric images captured in real low light environments and in partial occlusion. The performance of DnCNN model is compared with that of total variation denoising. Experimental results demonstrate that DnCNN performs better than total variation denoising for polarimetric integral imaging in terms of signal-to-noise ratio and structural similarity index measure in low light environments as well as low light environments under partial occlusions. To the best of our knowledge, this is the first report of polarimetric 3D object visualization and restoration in low light environments and occlusions using DnCNN with integral imaging. The proposed approach is also useful for 3D image restoration in conventional (non-polarimetric) integral imaging in a degraded environment.
© 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Passive polarimetric imaging is an attractive imaging technique which plays an important role in object recognition, material classification, and segmentation [1–3]. Polarimetric imaging involves the measurement of polarimetric signatures of reflected light from an object surface which contains additional information of surface features of object. The polarimetric information can be utilized in object recognition and material classification because of its ability to discriminate objects based on polarimetric signatures of materials. The polarimetric information of the object can be extracted by calculating the Stokes parameters and degree of polarization (DoP) [4–5]. The Stoke parameters and DoP are sensitive to noise in captured images in low light due to low photon counts and cause low signal-to-noise ratio (SNR) which make object visualization, recognition and material classification tasks difficult. Therefore, improving object visualization and reducing the noise in low light polarimetric imaging is highly desirable.
Integral imaging (InIm) is a prominent 3D imaging techniques for improving the scene visualization in low light [6–8]. The 3D image of the scene in integral imaging is obtained by recording 2D images from multiple perspectives, then reconstructing the scene, either computationally or optically. The 2D images from multiple perspectives known as elemental images can be captured by using a single camera with a lenslet array, or a camera array, or a single moving camera [9–18]. The reconstruction of 3D images using multiple perspectives of 2D images allows for capturing of depth information, reduces the effect of partial occlusion in front of the scene, and improves the performance of low light imaging due to being optimal in the maximum likelihood sense [6,7].
In this paper, we propose object visualization in low light environments under partial occlusions using denoising convolutional neural network (DnCNN) [19] with passive polarimetric integral imaging in low light illuminations. It has been shown that InIm can perform well in low light conditions [20–23]. The DnCNN is trained based on the physical model of the image capture in degraded environment to enhance the visualization of polarimetric imaging where simulated noisy polarimetric images are used in the training process. We show by optical experiments that 3D polarimetric imaging using DnCNN can be used to improve the visualization and reduce the noise disturbances in low light as well as low light plus partial occlusions. The performance of our proposed technique is presented by measuring the SNR and structural similarity index measure (SSIM) [24] of the 2D and 3D polarimetric images of the object scene in different low light illumination conditions, and in low light under partial occlusions. Our experiments indicate that using DnCNN, 3D polarimetric integral imaging provides higher SNR and higher SSIM as compared to the 2D polarimetric imaging in degraded environments. Furthermore, we compare the proposed 3D polarimetric imaging using DnCNN with 3D polarimetric imaging using total variation (TV) denoising [25] in terms of SNR and SSIM. For the experiments reported in this paper, the quantitative results show that recovered polarimetric images using DnCNN outperform the recovered polarimetric images using TV denoising in all degraded environments considered. The proposed approach is also useful for 3D image restoration in conventional (non-polarimetric) integral imaging in degraded environment such as low light illumination and occlusion.
2. Polarimetric integral imaging
2.1 Polarimetric imaging
Polarization of light is characterized by the relationship between the temporal average of magnitude and phase of two independent orthogonal electric field components. When the electric field components of electromagnetic wave oscillates only in a single direction, the light becomes linearly polarized. For a linearly polarized light propagating along the z direction, the electric field components of electromagnetic wave in the x and y directions are in phase. Stokes parameters are used to describe the polarization state of light. The Stokes parameters can be calculated using the set of captured polarimetric images [I0°, I45°, I90°, I135°, I45°,π/2, I135°,π/2] with multiple polarization directions as shown in Fig. 1. Iθ is the intensity of polarized light recorded when the linear polarizer in front of the imaging sensor is placed at an angle of θ with respect to the y-axis, and Iθ, π/2 is the intensity recorded after inserting a quarter wave plate (QWP) in addition to the linear polarizer [4–5]:
2.2 3D Integral imaging
In 3D integral imaging, the intensity and angular or directional information of light from a 3D scene is recorded by a single camera with a lenslet array or a camera array or a single camera on moving platform along the x and y directions. Each camera in the array or a camera position on a moving platform has a different location which records the image of 3D scene from different perspectives corresponding to the camera location. The recorded 2D image from multiple perspectives is referred to as an elemental image. The advantages of using 3D integral imaging over averaging of 2D imaging in noisy circumstances is that integral imaging allows the 3D information of object to be observed and analyzed by generating a series of depth images at an arbitrary distance by applying the computational reconstruction method to the multiple 2D elemental images. Moreover, the reconstructed 3D image in low light environments is optimal in the maximum likelihood sense at the corresponding reconstruction depth [6].
In our experiment, the polarimetric integral imaging is performed by moving a digital sCMOS (Hamamatsu C11440-42U) camera on two axes translational stage. A linear polarizer in front of camera is used to record the polarimetric information of 3D object. The pickup process of integral imaging is shown in Fig. 2(a) and the reconstruction of 3D imaging is done via synthetic aperture integral imaging (SAII) by using multiple perspectives of 2D elemental images as shown in Fig. 2(b) [26].
Fig. 2. Principle of 3D integral imaging. (a) Pickup process of integral imaging, and (b) computational reconstruction using synthetic aperture integral imaging (SAII).
The computational 3D polarimetric imaging of the scene at depth z is reconstructed according to the following equation [26]:
In Eq. (3), M and N are the total number of elemental images in horizontal (H) and vertical (V) directions, respectively. (x, y) is the pixel index and O(x, y) is the overlapping pixel number on (x, y). Iθm,n is the set of polarimetric elemental images with multiple directions θ [0°, 45°, 90°, 135°] while the subscripts m, n represent the location of the elemental image, and px and py are the camera pitch size in x- and y- directions, respectively. Lx and Ly are the total number of pixels in each column and row of images. cx × cy is the pixel size of camera, z is the pickup distance between the scene and the camera, f is the focal length of camera lens, and ε is the additive camera noise.
3. Experimental results and discussion
3.1 Experimental methods
In the experiments, elemental images were recorded by moving an sCMOS camera assembled with linear polarizer in front of the sensor on two axes translational stage. The scene which consists of toy cars and trucks was located approximately 2 m away from the camera set up. The focal length of camera lens is 50 mm and the sensor size of camera is 2048(H) × 2048(V) with pixel size 6.5µm×6.5 µm. In total, 9 (3(H) × 3(V)) elemental images were recorded with pitch size of 30 mm in x and y- directions using the sCMOS camera. The conventional 2D elemental images and 3D reconstructed images using SAII are shown in Fig. 3. Figure 3(a), Fig. 3(d) and Fig. 3(g) show the central 2D elemental images in high illumination, low illumination, and low illumination under partial occlusions, respectively. Two different reconstruction depth planes at z = 1.8 m (car) and at z = 2.1 m (truck) are shown in Fig. 3(b-c), Fig. 3(e-f), and Fig. 3(h-i), in high illumination, low illumination and low illumination under partial occlusions, respectively. The advantage of using 3D integral imaging is that it uses both angular and intensity information and provides a series of depth images at any desired distance which reduces the effects of partial occlusion in front of the scene [see Fig. 3(h-i)] and to segment out the objects of interest from the backgrounds. The estimated photons per pixel under low light environments is 2.5 [8,21].
Fig. 3. Conventional 3D integral imaging in high illumination, low illumination and low illumination under partial occlusions. Top row presents high illumination experiments. (a) Central 2D elemental image, (b) reconstructed 3D image at z = 1.8m, and (c) reconstructed 3D image at z = 2.1m in high illumination. Middle row presents low light illumination experiments. (d) Central 2D elemental image, (e) reconstructed 3D image at z = 1.8m, and (f) reconstructed 3D image at z = 2.1m under low light environments. Bottom row presents experiments of scene in low light environments under partial occlusions. (g) Central 2D elemental image, (h) reconstructed 3D image at z = 1.8m, and (i) reconstructed 3D image at z = 2.1m in low light environments under partial occlusions. The estimated photons per pixel in low light is 2.5.
Feed forward denoising convolutional neural networks (DnCNN) [19] is used to improve the visualization of polarimetric imaging in degraded environments. The input of DnCNN is a simulated low light noisy polarimetric image y = αx + v, where y represents the noisy observation of ideal polarimetric imaging x (ground truth), α represents the degradation parameter and v is the additive Gaussian noise from camera sensor. The denoising of an image is considered as plain discriminative learning problem, i.e. separating the noise from a noisy image by feed-forward convolutional neural network. It is designed to predict the noise at each pixel of the residual image (difference between noisy image and latent clean image) rather than directly outputting the denoised image. The residual network uses many residual units to predict the output [27]. However, the residual learning formulation in the DnCNN uses a single residual unit to predict the residual image [19]. The residual learning formulation is used to train the residual mapping R(y) between the deformation map (residual image) and the degraded input, then we have x = y - R(y). The loss function l(θ) in Eq. (4) below is the averaged mean square error between the residual images and the estimated one from degraded input:
To train the DnCNN, we generated the synthetic (simulated) low light polarimetric images. This was done by applying the constant multiplicative degradation parameter (α) with additive Gaussian noise (v ∼ N (0, σ2)) to the polarimetric image in high illumination conditions (y = αx + v). The polarimetric integral images in high illumination are shown in Fig. 4(a-c). The high illumination 2D DoLP image is shown in Fig. 4(a), and the corresponding 3D DoLP images of the scene reconstructed at depth z = 1.9 m is shown in Fig. 4(b), and at depth z = 2.1 m is shown in Fig. 4(c). The constant multiplicative degradation parameter (α) is drawn randomly from the uniform distribution ranging from 0 to 1 and the standard deviation for the Gaussian noise is chosen from the range [0.1 to 0.9] to train the DnCNN model. The simulated low light reconstructed polarimetric images are shown in Fig. 4(d-f). The low light noisy 2D DoLP image is shown in Fig. 4(d), and the corresponding 3D DoLP images of the noisy low light scene reconstructed at depth z = 1.9 m is shown in Fig. 4(e), and at depth z = 2.1 m is shown in Fig. 4(f).
Fig. 4. Polarimetric 3D integral imaging results used to train the denoising CNN (DnCNN). Top row is the results in high illumination. (a) Central 2D DoLP image of scene, (b) 3D DoLP images reconstructed at depth z = 1.9 m, and (c) 3D DoLP images reconstructed at depth at z = 2.1 m all in high illumination. Bottom row is the results in low illumination including sensor noise. (d-f) Synthetic low light polarimetric images for training the neural network. (d) Noisy 2D DoLP image of low light scene, (e) corresponding 3D DoLP image reconstructed at depth z = 1.9 m, and (f) 3D DoLP image reconstructed at z = 2.1 m.
The architecture of DnCNN shown in Fig. 5 is used to train the residual mapping R(y) consists of 17 convolutional layers. The first convolutional layer uses a rectified linear unit (ReLU) activation function at its output without batch normalization and uses a filter of size 3×3×c, with c being the number of color channels. The following 15 convolutional layers employ both a ReLU activation function as well as batch normalization (BN) using a filter size of 3×3×64. The final convolutional layer has a filter size of 3×3×64 but does not use either ReLU activation or BN. The input to the network is the noisy polarimetric image, and the output is the residual noise image. Each convolutional layer uses a stride of 1 and the padding is set ‘same’ to maintain the input image size throughout the network [19]. Finally, the recovered image is obtained by subtracting the deformation map (residual image) from the distorted image. We trained two different DnCNN models in order to compare the denoising of polarimetric images in 2D and 3D DoLP imaging. In the first approach, one DnCNN model is trained on the simulated low light 2D DoLP images. In the second approach, a DnCNN model is trained on the simulated low light 3D DoLP reconstructed images. 100 passive polarimetric images were collected in high illumination conditions without occlusion, then these images were used to produce simulated noisy images. 90 images of one scene were used for training, and 10 images of another scene were used as a validation dataset for hyperparameter tuning. The same number of 3D DoLP images are used to train and validate the DnCNN model for fair comparison. The model was trained with Adam optimizer, learning rate of 1×10−3 and batch size of 16. The model is trained for 100 epochs.
Fig. 5. The architecture of denoising convolutional neural network (DnCNN) . ReLU is rectified linear unit activation function. BN is batch normalization.
3.2 Results and discussion
The test dataset is captured experimentally from real low light environments scenes using polarimetric integral imaging system as discussed in Section 3.1. To test the performance of the trained DnCNN model, we have recorded 40 passive polarimetric images in different low light conditions, both with (10 passive polarimetric images) and without partial occlusions (30 passive polarimetric images). A sample polarimetric 2D DoLP image in real low light is shown in Fig. 6(a). The estimated photons per pixel is calculated as 8 [8,21]. The captured image has low signal-to-noise ratio due to low light illumination conditions and camera noise. The low SNR in passive polarimetric images in low light environments is evident in DoLP calculation due to the presence of camera noise in processing of Stokes parameters which deteriorates the polarization information of the scene. The impact of noise in the noisy 2D DoLP image is mitigated in the reconstructed 3D DoLP image, as can be seen at depth z = 1.9 m, and at depth z = 2.1 m as shown in Fig. 6(b) and Fig. 6(c), respectively.
Fig. 6. Polarimetric images captured optically in low light environments (8 photons/pixel) for testing the performance of DnCNN model. (a) 2D DoLP image of the low light scene, (b-c) corresponding 3D DoLP images of the scene reconstructed at z = 1.9 m, and at z = 2.1 m, respectively. (d-f) Recovered polarimetric images using TV denoising. (d) Recovered 2D DoLP image using TV denoising, corresponding 3D DoLP images of the scene reconstructed (e) at z = 1.9 m, and (f) at z = 2.1 m. (g-i) DnCNN recovered noise reduced polarimetric images. (g) DnCNN recovered 2D DoLP image, corresponding DnCNN recovered 3D DoLP images of the scene reconstructed (h) at z = 1.9 m, and (i) at z = 2.1 m. DnCNN performs better than TV denoising to recover DoLP in low light conditions.
For comparison to our proposed method, TV denoising algorithm is applied to the DoLP images to reduce the noise and enhance the visualization of polarimetric scene. Figure 6(d-f) shows the recovered polarimetric 2D DoLP image [Fig. 6(d)], and 3D DoLP images at depth z = 1.9 m [Fig. 6(e)] and at depth z = 2.1 m [Fig. 6(f)] after using TV denoising. Figure 6(g-i) shows the results of enhancement of polarimetric visualization using DnCNN model in low light environments. The amount of noise in the recovered 2D DoLP image using DnCNN is reduced in comparison to 2D DoLP image [see Fig. 6(a)] and recovered 2D DoLP image using TV denoising [see Fig. 6(d)], but still there is a residual noise present in the recovered 2D DoLP scene which degrades the polarimetric image. However, the recovered 3D DoLP images using DnCNN, shown in Fig. 6(h) [at depth z = 1.9 m] and Fig. 6(i) [at depth z = 2.1 m], show better enhancement of polarimetric visualization in low light environments as compared to the recovered 2D DoLP image using DnCNN and 3D DoLP images using TV denoising. The recovered 3D DoLP images using DnCNN reduces the residual noise as well as preserves the detailed polarization information.
Furthermore, DnCNN model was tested on polarimetric images in different low light conditions (2-20 photons/pixel), and low light under partial occlusions for the quantitative analysis of the performance of trained DnCNN model. Polarimetric scene in low light environments is shown in Fig. 7(a-c). The recovered DoLP images using TV denoising and DnCNN in low light conditions are shown in Fig. 7(d-f) and Fig. 7(g-i), respectively. The recovered DoLP images in low light illumination conditions (20 photons/pixel) using TV denoising are shown in Fig. 7(d) for 2D DoLP image, and the corresponding 3D DoLP images in Fig. 7(e) at depth z = 1.8 m, and in Fig. 7(f) at depth z = 2.1 m. The recovered DoLP images using DnCNN in low light illumination conditions (20 photons/pixel) are shown in Fig. 7(g) for 2D DoLP image, and the corresponding 3D DoLP image in Fig. 7(h) for depth z = 1.8 m, and in Fig. 7(i) for depth z = 2.1 m.
Fig. 7. Polarimetric 2D, and 3D integral imaging experimental results of a scene in low light (20 photons/pixel) for testing the performance of DnCNN model are presented. (a) 2D DoLP image of the scene, (b-c) the corresponding 3D DoLP images of the scene reconstructed at z = 1.8 m, and at z = 2.1 m, respectively. (d-f) Recovered polarimetric images using TV denoising. (d) 2D DoLP image, and corresponding 3D DoLP images of the scene reconstructed (e) at z = 1.8 m, and (f) at z = 2.1 m. (g-i) Recovered polarimetric images using DnCNN. (g) 2D DoLP image, and corresponding 3D DoLP images of the scene reconstructed (h) at z = 1.8 m, and (i) at z = 2.1 m. DnCNN performs better than TV denoising to recover DoLP in low light conditions.
The performance of trained DnCNN model is tested in degraded environment of low light plus partial occlusions and results are compared with results of TV denoising. Polarimetric scene in low light environments (10 photons/pixel) under partial occlusions is shown in Fig. 8(a-c). The recovered polarimetric images using TV denoising are shown Fig. 8(d) for 2D DoLP image, in Fig. 8(e) for 3D DoLP image at z = 2.0 m, and in Fig. 8(f) for 3D DoLP image at z = 2.3 m. The recovered polarimetric images using DnCNN are shown Fig. 8(g) for 2D DoLP image, in Fig. 8(h) for 3D DoLP image at z = 2.0 m, and in Fig. 8(i) for 3D DoLP image at z = 2.3 m. The 3D DoLP images reduce the effect of partial occlusions in front of the scene and segment out the objects of interest from the background.
Fig. 8. Experimental results for polarimetric images of scene in low light plus partial occlusion for evaluating the performance of DnCNN model. (a) 2D DoLP image of the scene, (b-c) 3D DoLP images of the scene reconstructed at z = 2.0 m, and at z = 2.3 m. (d-f) Recovered polarimetric images using TV denoising. (d) 2D DoLP image, 3D DoLP images of the scene reconstructed (e) at z = 1.8 m, and (f) at z = 2.1 m. (g-i) Recovered noise reduced polarimetric images using DnCNN. (g) 2D DoLP image, and 3D DoLP images of the scene reconstructed (h) at z = 2.0 m, and (i) at z = 2.3 m. The estimated photons/pixel is 10. DnCNN performs better than TV denoising to recover DoLP in low light and partial occlusion conditions.
For each polarimetric scene under different low light illumination, the quantitative metrics of SNR and structure similarity index measure (SSIM) are calculated to evaluate the performance of DnCNN in different environmental degraded conditions. The SNR of the polarimetric (object signal) area compared to the non-polarimetric (background noise) area is measured and defined as $\textrm{SNR = }\frac{{({\mu _s} - {\mu _b})}}{{\sqrt {\sigma _s^2 + \sigma _b^2} }}$, where µs and µb are the mean of polarimetric area (signal) and non-polarimetric area (background), respectively, and σs2 and σb2 are the variances of polarimetric area and non-polarimetric area, respectively. The same area is selected from the signal and background region in both the 2D and 3D DoLP images. The background region is chosen as the area which contains the lowest pixel value. The performance of DnCNN in different degraded conditions is compared with TV denoising in term of SNR. Table 1 shows the SNR of different scenes under different degraded condition such as different low light condition (estimated photons/pixel are 2-20), and different low light environments under partial occlusions. The quantitative result of this comparison in Table 1 indicates that the recovered polarimetric images using DnCNN produce SNR that is higher than the recovered polarimetric images using TV denoising in all degraded environments for 2D and 3D DoLP images.
Table 1. Experimental results for SNR measurement of recovered polarimetric images of scenes in different low light levels (2 - 20 photons/pixel)
SSIM is used to measures the structural similarity between original image (Polarimetric image in high illumination) and the polarimetric images of the test scene (Polarimetric image in low light light), and is related to the distortion of visual sensing. SSIM of two different images a and b is defined as $\textrm{SSIM(}a\textrm{,}b\textrm{) = }\frac{{(2{\mu _a}{\mu _b} + {c_1})(2{\sigma _{ab}} + {c_2})}}{{({\mu _a}^2 + {\mu _b}^2 + {c_1})({\sigma _a}^2 + {\sigma _b}^2 + {c_2})}}$, where µa, and µb are the mean value for the reference image patch a (area of polarimetric object of scene in high illumination) and test image patch b (area of polarimetric object of scene in low light), σa and σb are the standard deviation of image patch a and b, respectively and σab is the covariance of a and b. c1 and c2 are the adjustable parameter defined as c1= (k1L)2 and c2= (k2L)2 with L being the largest pixel value, and k1=0.01, and k2=0.03 [24]. Table 2 shows the SSIM of different scenes under different low light conditions (estimated photons/pixel are 2-20).
Table 2. Experimental results for SSIM values of recovered polarimetric images of scenes in different low light levels (2 - 20 photons/pixel)
Figure 9(a) and 9(b) show the graphs of experimental results of SNR and SSIM, respectively, of the polarimetric images of scenes in different low light conditions and recovered polarimetric image using TV denoising and DnCNN with respect to photons/pixel. The quantitative analysis in Table 1 and Table 2 indicates that DnCNN with 3D polarimetric integral imaging outperforms the DnCNN with 2D DoLP images, and the recovered 2D DoLP and 3D DoLP images using TV denoising in all degraded environments. Therefore, for the experiments we performed and presented here, the proposed method of DnCNN with polarimetric integral imaging is able to reduce the noise and enhance the visibility of polarimetric information in all types of degraded environments such as low light and low light under partial occlusion.
Fig. 9. (a) SNR and (b) SSIM obtained by experimental results for 2D and 3D polarimetric images and recovered polarimetric images using TV denoising and DnCNN of the scene under different low light levels.
4. Conclusions
In summary, we have used a denoising convolution neural network (DnCNN) to enhance the visibility of polarimetric 3D integral imaging in degraded environments such as 1) low light illumination, and 2) low light under partial occlusions. Low light conditions are considered in the range of 2-20 photons/pixel. Residual learning formulation is adopted to separate the noise due to photon-starved conditions in the captured polarimetric images and enhance the visualization in low light conditions. The performance of proposed DnCNN with 3D polarimetric imaging is compared to the DnCNN with 2D polarimetric imaging as well as polarimetric imaging using TV denoising in terms of SNR and SSIM in different degraded environments. Quantitative comparison between the DnCNN and TV denoising with polarimetric imaging in degraded environments indicates that the proposed DnCNN technique with polarimetric integral imaging outperforms the TV denoising technique with polarimetric integral imaging. In our experiments, the recovered polarimetric images using DnCNN with 2D polarimetric imaging contain residual noise which distort the polarimetric measurements. However, DnCNN with 3D polarimetric imaging effectively reduces the noise due to photon starved conditions and camera noise, and enhances polarimetric 3D visualization. For the experiments we performed and presented here, the results indicate that the proposed method is able to mitigate noise as well as the partial occlusion in the scene in low light environments. To the best of our knowledge, this the first report of polarimetric 3D visualization in various degraded environments using DnCNN with 3D polarimetric integral imaging. The proposed environmental degradation physical model based denoising convolution neural network approach is also useful for 3D image restoration in conventional (non-polarimetric) Integral Imaging.
Funding
Office of Naval Research (N000141712405, N00014-17-1-2561, N000142012690); Air Force Office of Scientific Research (FA9550-18-1-0338, FA9550-21-1-0333).
Acknowledgements
B. Javidi acknowledges support by Air Force Office of Scientific Research (FA9550-18-1-0338, FA9550-21-1-0333); and Office of Naval Research (N000141712405, N00014-17-1-2561, N000142012690). T. O’Connor acknowledges the Department of Education through the GAANN Fellowship. We wish to thank Hamamatsu Photonics K. K. for the C11440-42U camera.
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. L. B. Wolff, “Polarization-based material classification from specular reflection,” IEEE Trans. Pattern Anal. Mach. Intell. 12(11), 1059–1071 (1990). [CrossRef]
2. J. S. Tyo, D. L. Goldstein, D. B. Chenault, and J. A. Shaw, “Review of passive imaging polarimetry for remote sensing applications,” Appl. Opt. 45(22), 5453–5469 (2006). [CrossRef]
3. K. T. Thilak, C. D. Creusere, and D. G. Voelz, “Passive polarimetric imagery-based material classification robust to illumination source position and viewpoint,” IEEE Trans. Image Process. 20(1), 288–292 (2011). [CrossRef]
4. G. P. Konnen, Polarized light in nature (Cambridge University, 1985).
5. M. Born and E. Wolf, Principles of Optics: Electromagnetic Theory of Propagation, Interference and Diffraction of Light (Cambridge University, 1999).
6. B. Tavakoli, B. Javidi, and E. Watson, “Three-dimensional visualization by photon counting computational integral imaging,” Opt. Express 16(7), 4426–4436 (2008). [CrossRef]
7. A. Markman, X. Shen, and B. Javidi, “Three-dimensional object visualization and detection in low light illumination using integral imaging,” Opt. Lett. 42(16), 3068–3071 (2017). [CrossRef]
8. A. Stern, D. Aloni, and B. Javidi, “Experiments with three-dimensional integral imaging under low light levels,” IEEE Photonics J. 4(4), 1188–1195 (2012). [CrossRef]
9. G. Lippmann, “Epreuves reversibles donnant la sensation du relief,” J. Phys. 7(1), 821–825 (1908). [CrossRef]
10. S. H. Hong, J. S. Jang, and B. Javidi, “Three-dimensional volumetric object reconstruction using computational integral imaging,” Opt. Express 12(3), 483–491 (2004). [CrossRef]
11. N. Davies, M. McCormick, and L. Yang, “Three-dimensional imaging systems: a new development,” Appl. Opt. 27(21), 4520–4528 (1988). [CrossRef]
12. F. Okano, H. Hoshino, J. Arai, and I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. 36(7), 1598–1603 (1997). [CrossRef]
13. B. Javidi, X. Shen, A. Markman, P. L. Carmona, A. Martinez-Uso, J. M. Sotoca, F. Pla, M. Martinez-Corral, G. Saavedra, Y. Huang, and A. Stern, “Multidimensional Optical Sensing and Imaging System (MOSIS): From macroscales to microscales,” Proc. IEEE 105(5), 850–875 (2017). [CrossRef]
14. J. Liu, D. Claus, T. Xu, T. Keßner, A. Herkommer, and W. Osten, “Light field endoscopy and its parametric description,” Opt. Lett. 42(9), 1804–1807 (2017). [CrossRef]
15. G. Scrofani, J. Sola-Pikabea, A. Llavador, E. Sanchez-Ortiga, J. C. Barreiro, G. Saavedra, J. Garcia-Sucerquia, and M. Martinez-Corral, “FIMic: design for ultimate 3D-integral microscopy of in-vivo biological samples,” Biomed. Opt. Express 9(1), 335–346 (2018). [CrossRef]
16. J. Arai, E. Nakasu, T. Yamashita, H. Hiura, M. Miura, T. Nakamura, and R. Funatsu, “Progress overview of capturing method for integral 3-D imaging displays,” Proc. IEEE 105(5), 837–849 (2017). [CrossRef]
17. M. Yamaguchi, “Full-parallax holographic light-field 3-D displays and interactive 3-D touch,” Proc. IEEE 105(5), 947–959 (2017). [CrossRef]
18. M. Martínez-Corral and B. Javidi, “Fundamentals of 3D imaging and displays: a tutorial on integral imaging, light-field, and plenoptic systems,” Adv. Opt. Photonics 10(3), 512–566 (2018). [CrossRef]
19. K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, “Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising,” IEEE Trans. Image Process. 26(7), 3142–3155 (2017). [CrossRef]
20. A. Carnicer and B. Javidi, “Polarimetric 3D integral imaging in photon starved conditions,” Opt. Express 23(5), 6408–6417 (2015). [CrossRef]
21. X. Shen, A. Carnicer, and B. Javidi, “Three-dimensional polarimetric integral imaging under low illumination conditions,” Opt. Lett. 44(13), 3230–3233 (2019). [CrossRef]
22. K. Usmani, T. O’Connor, X. Shen, P. Marasco, A. Carnicer, D. Dey, and B. Javidi, “Three-dimensional polarimetric integral imaging in photon-starved conditions: performance comparison between visible and long wave infrared imaging,” Opt. Express 28(13), 19281–19294 (2020). [CrossRef]
23. K. Usmani, G. Krishnan, T. O’Connor, and B. Javidi, “Deep learning polarimetric three-dimensional integral imaging object recognition in adverse environmental conditions,” Opt. Express 29(8), 12215–12228 (2021). [CrossRef]
24. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process. 13(4), 600–612 (2004). [CrossRef]
25. L. I. Rudin, S. Osher, and E. Fatemi, “Nonlinear total variation-based noise removal algorithms,” Phys. D 60(1-4), 259–268 (1992). [CrossRef]
26. J. S. Jang and B. Javidi, “Three-dimensional synthetic aperture integral imaging,” Opt. Lett. 27(13), 1144–1146 (2002). [CrossRef]
27. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 770–778 (2016).
