1. Introduction

Polarization is a degree of freedom of light and describes the path along the oscillation of light’s electric fields. Unfortunately, our eyes are blind to the polarization state. Polarization imaging systems instead reveal it over a scene of interest, providing valuable information about imaged environments, such as shape, texture, and chirality of reflecting surfaces [1,2]. This information is usually absent in the other two fundamental properties of light: intensity and spectral content. Its importance has recently increased with the rapid development of image recognition using artificial intelligence. This is because polarization information may be used for image recognition in addition to primary color channels captured with conventional cameras; this new input channel can allow for improved recognition accuracy and extended applications. Therefore, polarization imaging has found an extensive number of applications, ranging from biology to remote sensing and machine vision [3–12].

To achieve polarization imaging, various optical systems have been developed over the past decades [13–20]. They are generally classified as division-of-time (acquisition by multiple shots), division-of-amplitude (acquisition by multiple optical paths), or division-of-focal-plane (DoFP: acquisition at a focal plane array) [1]. These systems rely on traditional single-eye imaging optics (one main lens is responsible for object imaging function) with rotating polarizers, cascaded optical elements, or micropolarizer-integrated sensor arrays [1]. A DoFP system requires less complicated optics and enables the creation of more compact devices compared to the two other systems [17–20], making it the most prevalent approach for polarization imaging today. Polarization cameras based on such conventional systems, however, have two common issues. The first issue is low sensitivity due to the low light utilization efficiency of absorptive polarizers or cascaded optical elements. The other is that there is a constraint in reducing device size. This is due to the fact that such conventional systems require at least an optical-path length equivalent to the focal length of the main lens. These issues could be critical, especially in recently emerging applications, such as vision systems for autonomous vehicles, drones, and smart phones, in which available light is limited and strict size constraints exist.

Optical metasurfaces, arrays of subwavelength-spaced photonic nanostructures [21–26], can offer a route to address the above issues. As optical metasurfaces can be designed as a flat optical component with unique and multiple functionalities [27–36], they can enhance imaging capabilities while reducing the required optics [37–46]. It has been demonstrated that metasurface-based systems with a filter-free scheme can overcome the theoretical efficiency limit with conventional filter-based polarization or color imaging systems [43,44,46]. However, since they still rely on a traditional single-eye imaging scheme, the size of the devices based on such systems is fundamentally limited by imaging optics (a lens); thus, equivalent to the above conventional systems.

We propose a new imaging system based on compound-eye metasurface optics for creating a high-sensitivity, ultra-thin polarization camera. Compound-eye optical systems combined with computational techniques have been proposed as analogues to biological vision systems (arthropod eyes) [47–49]. They can dramatically reduce the optical-path length required for imaging, resulting in a more compact device than a conventional single-eye-based system. In this study, we designed metasurface-based lenses (metalenses) with the ability of polarization splitting and imaging within a single layer and applied them to computational compound-eye imaging. Since this approach is based on filter-free imaging with an ultra-thin compound-eye scheme, it can overcome both sensitivity and device-miniaturization limitations of conventional polarization cameras. As a proof of concept, we designed and fabricated a compound-eye, polarization-splitting metalens operating in the visible wavelength and demonstrated its effectiveness in polarization imaging. Compared with conventional DoFP systems, the proposed system enhances the amount of detected light by a factor of ∼2, while dramatically reducing the device size. These features are promising for achieving high-sensitivity polarization imaging with an ultra-compact device.

2. Polarization imaging system based on a compound-eye metalens

A conventional DoFP polarization camera is composed of one main lens and an image sensor integrated with a micropolarizer array, as shown in Fig. 1(a). The lens focuses the incident light from an object onto a sensor plane, while the micropolarizers are used to extract linearly polarized light along four axes from the light since the sensor detectors are unable to distinguish polarization. Three Stokes parameters S_0-2 can be derived from the four polarization bases. Therefore, this camera can capture 2D images including polarization information such as angle of polarization [AoP: (1/2)×arctan(S₂/S₁)] and degree of linear polarization [DoLP: (S₁²+S₂²)^1/2/S₀] in addition to intensity information (S₀) [1]. However, it is clear that the light utilization efficiency is limited up to 50% due to the micropolarizers with different transmission axes, and the device thickness is limited by the focal length of the lens.

 

Fig. 1. Polarization imaging based on compound-eye metasurface optics. (a, b) Cross-sectional diagrams of (a) conventional division-of-focal-plane (DoFP) polarization camera and (b) metasurface-based polarization camera. (c) Image-formation model of polarization imaging with compound-eye metalens. For clarity, letter color in images represents polarization basis.

Download Full Size | PDF

Figure 1(b) shows the schematic of our imaging system, which can overcome the above limitations. Our system consists of an array of polarization-splitting metalenses and a conventional image sensor. To extract S_0-2, the array consists of sets of two metalenses that split linear polarization bases along the 0° (x)/90° (y) directions and -45°/+45°directions. When polarized light from a scene is incident on this architecture, the polarization is first sorted by each metalens, and a vast number of images corresponding to four polarization bases are then formed on the sensor. Although each captured image has a low spatial resolution, a computational technique for compound-eye imaging allows dramatic resolution enhancement, enabling the creation of high-resolution polarization images. This relies on the fact that the optical function can be expressed as a simple linear system. Figure 1(c) shows the system model of the compound-eye metalens function. An original image is first classified into four images that correspond to four polarization bases, which are then copied, shifted, and downsized due to the compound-eye architecture, and the resulting images are finally captured on the sensor. Since each unit image has different information due to parallax, a high-resolution image can be reconstructed from the captured image by solving the inverse problem [47]. Considering the optical system of a one-dimensional model for simplicity, the captured image vector Y can be modeled as Y = ΦX, where X is the original image vector and Φ is the system matrix, represented in the form of

In this system, almost all the incident light is used for polarization imaging because the operation is based on polarization splitting instead of polarization filtering. Therefore, assuming that the quantum efficiency is the same for sensors, 2-fold enhancement of the detected signal is expected. This greatly improves sensitivity if the metalenses perform sufficiently for polarimetry and the computational reconstruction errors are negligibly small. In addition, the focal length of each metalens is ∼N times shorter than that of the single-eye imaging architecture with a comparable field of view. This results in a compact device whose thickness is ∼1/N that of a conventional single-eye-based camera. Therefore, a compound-eye metalens enables high-sensitivity polarization imaging with an extremely thin device. It is important to note that our system requires image-reconstruction calculations that are often based on the iteration method. For this reason, it is not unusual for the image-reconstruction time to exceed the acquisition time; this is in contrast to the DoFP system that directly acquires polarization information as pixel signals. Nevertheless, for applications that permit post-processing, our system is promising for eliminating both sensitivity and device-size constraints.

3. Metalens design and characterization

Based on the above concept, we first designed and characterized polarization-splitting metalenses operating in the visible wavelength. The metasurface platform we used is based on dielectric nanoposts with rectangular cross sections. Optical metasurfaces consisting of such structures can be designed as an optical wavefront control element with a degree of freedom of birefringence [33]. As shown in Figs. 2(a) and 2(b), we used silicon nitride (SiN) nanoposts as dielectric building blocks because they support metasurface functionalities in the visible wavelengths and are potentially compatible with CMOS process technologies for mass production [50].

 

Fig. 2. Design of polarization-splitting metalenses. (a) Side view and (b) top view of silicon nitride (SiN) nanopost with rectangular cross section on quartz substrate. Nanoposts along orthogonal coordinates were used to split 0°/90° polarization bases, and 45°-rotated nanoposts were used to split -45°/+45° polarization bases. (c) Ideal phase profiles to split two orthogonal bases of polarization and respectively focus them to left (x = -50 µm, y = 0 µm) and right centers (x = +50 µm, y = 0 µm) on focal plane. (d) Designed phase profiles with nanoposts. Only nanopost widths (w₁, w₂) = 120–280 nm were used to facilitate large-area-fabrication. (e) Optical image of fabricated metalenses. Polarization basis for each metalens is illustrated with colored arrows. Scale bar: 50 µm. (f) Scanning electron microscopy images of fabricated metalenses. Scale bars: 1 µm.

Download Full Size | PDF

We first investigated the transmission properties of SiN nanopost arrays on a quartz substrate under x- and y-polarized light illumination using a rigorous coupled wave analysis, where x and y represent the axes aligned with the nanopost geometry [see Fig. 2(b)]. We considered 700-nm-thick SiN nanoposts operating at a wavelength of 520 nm. This thickness ensures a sufficient phase range for controlling the optical wavefront for the operating wavelength over both types of polarized light but is still thin enough to facilitate fabrication. The lattice constant was set to 400 nm to prevent light diffraction on the transmission side. The simulated transmission properties for x- and y-polarized light as functions of nanopost widths w₁ and w₂ are summarized in Appendix A1, showing that the nanoposts can provide a high degree of independent phase control over both polarizations while preserving high transmittance. Using these properties, one can find the nanopost geometry required to match a pair of desired phases for both polarizations in a point-by-point manner (see the details in Appendix A1).

Based on these analyses, we then designed a metalens that forms images at different positions in a focal plane according to the polarization basis. The metalens was designed for a 100 × 200 µm² aperture, 500-µm focal length, and 520-nm operating wavelength. We also used only the nanoposts with w₁, w₂ = 120–280 nm for the design; this excludes tiny geometries (< ∼120 nm) from the designed pattern; thus, facilitating large-area-fabrication. Figure 2(c) shows the ideal phase profiles to split two orthogonal polarization bases (0°/90°) and respectively focus them to the left (x = -50 µm, y = 0 µm) and right centers (x = +50 µm, y = 0 µm) on a focal plane. These profiles mimic the phase profile φ of an off-axis lens, which can be written as

We then fabricated two metalenses on a quartz substrate using standard thin-film deposition, single-step electron-beam lithography, and etching processes. Briefly, a 700-nm-thick SiN layer was first deposited on a quartz substrate. After the deposition of a metallic mask layer, the nanopost pattern was transferred to the mask layer using electron-beam lithography and dry etching. The pattern was then transferred to the SiN layer by dry etching through the mask. The mask layer was finally removed by dry etching. An optical image of the two metalenses (0°/90° and -45°/+45° polarization-splitting metalenses) is shown in Fig. 2(e). The metalenses were arranged side by side (the arrows denote the polarization bases). Scanning electron microscopy (SEM) images of their small area are also shown in Fig. 2(f).

To experimentally characterize the metalenses, we first investigated their focusing performance by varying the polarization state. Specifically, we first characterized the focal length of the metalenses under plane light illumination (520-nm wavelength) using a custom-built optical microscope (see the optical setup details in Appendix A2) by sweeping its image plane. We confirmed that both metalenses concentrate light into spots at the designed focal length. We then imaged the focal plane of the metalenses under plane light illumination (520-nm wavelength) with different linear polarization states using the same setup. Figure 3(a) shows the measured intensity profiles for the four linear polarization states. The zoomed-in profiles on the focusing spots [indicated with the dotted squares in Fig. 3(a)] are summarized in Fig. 3(b). These profiles show the clear ability of the metalenses to sort and focus the polarized light as designed; for the 0°/90° (-45°/+45°) polarization-splitting metalenses, the 0° (-45°) polarized light is strongly focused to the left focal point, while the 90° (+45°) polarized light is focused to the opposite point. To gain further insights, we evaluated the focusing intensity at each focal point while varying the linear polarization state in 5° steps, as summarized in Fig. 3(c). The focusing intensity is defined as the total power of the light inside a radius equal to the distance from the focal central point to the first null point along the vertical axis in the intensity profile. The graph shows sinusoidal response in accordance with Malus’s law and high similarity between different focal points. More specifically, the extinction ratio (ER), defined by the ratio of the maximum to minimum values of a sinusoidal plot [18] averaged over the four focal points, was ∼19.2. This value is comparable to that (5∼90) for state-of-the-art micropolarizers [18–20,51–53], indicating the capability of the metalenses for reliable polarimetry.

 

Fig. 3. Polarization-dependent focusing with metalenses. (a) Measured intensity profiles on metalenses’ focal plane under plane light illumination (520-nm wavelength) with different linear polarization states. Polarization basis is illustrated with colored arrows. Scale bars: 50 µm. (b) Zoomed-in intensity profiles on focusing spots indicated with dotted squares in (a). Scale bars: 5 µm. (c) Focusing intensity at each focal point as function of incident angle of polarization (AoP).

Download Full Size | PDF

We estimated the potential performance achievable in our metasurface platform for polarimetry using full-field simulations. Unfortunately, we could not directly simulate the performance achievable in our metalens design because of its extremely large simulation size (i.e. high calculation cost). Instead, we simplified the design by excluding the lens function and designed and simulated the polarization splitters with the same metasurface platform (see Appendix A3). This allows us to simply estimate the potential performance of our metasurface platform for polarimetry (i.e. achievable ER) while greatly reducing the calculation costs. The simulations show that the platform can achieve an ER of ∼100, implying that there is still room for improving the performance of the actual metalens. The simulations also show that the design using SiN nanoposts with a full range of width parameters (nanopost widths: w₁, w₂ = 80–350 nm) exhibits better performance than that using nanoposts with a limited parameter range (nanopost widths: w₁, w₂ = 120–280 nm). This is due to the fact that the phase sets available in the nanoposts with w₁, w₂ = 80–350 nm show better matching to the ideal ones. Although such nanostructures require more delicate fabrication processes that can support even finer geometries, using them for the design can be one path to improving the metalens performance for polarimetry. Improving fabrication for the current design (w₁, w₂ = 120–280 nm) may also be promising for performance improvement because the nanopost width deviation directly leads to a phase error, often degrading the metasurface performance [33].

Next, we explored how the metalenses behave when polarized light from a photographic scene is incident on them. We prepared a printed mandrill image covered with a linear polarizer and imaged it through the metalenses at the 520-nm wavelength while rotating the polarizer (see the optical setup details in Appendix A2). Figure 4 summarizes the captured images for the four linear polarization states. All the captured images taken while varying the linear polarization state in 5° steps are also shown as a video in Visualization 1. It is clear that four copies of the mandrill image corresponding to the four polarization bases simultaneously formed on the focal plane, demonstrating the metalenses’ ability to form images while splitting the polarization bases. We also measured the transmittances of the metalenses under plane light illumination while changing the incident polarization in 5° steps. The total transmittance (i.e. light utilization efficiency) averaged over the two metalenses and over all the incident polarizations was 68.4%. This value is lower than the expected transmittance (79.7%: the value averaged over the simulated transmittances of all the unit nanoposts comprising the metalenses). This can be attributed to reflection from the substrate backside (∼3%) as well as undesired scattering by the roughness of the etched SiN nanoposts and substrate. Nevertheless, the measured transmittance greatly exceeds the 50% theoretical limit of a filter-based polarization camera. Considering that the total transmittance of a typical micropolarizer array for DoFP polarization cameras is only 30 to 35% [18], the metalenses improve efficiency by a factor of ∼2.

 

Fig. 4. Polarization-dependent imaging with metalenses. Polarizer-covered mandrill image was formed through metalenses at 520-nm wavelength while rotating polarizer. Polarization basis is illustrated with colored arrows. Scale bars: 50 µm.

Download Full Size | PDF

4. Polarization imaging

We applied these metalenses to the compound-eye architecture and demonstrated their capability for creating a high-sensitivity, ultra-thin polarization camera. We constructed an optical setup based on an optical microscope, as shown in Fig. 5(a). This setup was designed to virtually simulate the proposed system by transferring the formed image on the metalenses’ focal plane to a monochrome image sensor by lens optics. A band-pass filter (520 ± 5 nm) was inserted into the microscope to evaluate the performance at the designed wavelength. We prepared a 1.0 × 1.0 mm² compound-eye metalens that simultaneously forms 10 × 10 images on the sensor (each metalens was designed with the same parameters described in Figs. 2–4); thus, the device thickness is about 1/10 that of a single-eye-based camera with a comparable field of view. As a target object, a printed mandrill image covered with four polarizer films with different transmission axes [see the arrows in the inset of Fig. 5(a)] was placed 54.6 mm from a metalens and illuminated by unpolarized light from behind. The pixel size of the captured images (magnified by the objective lens) was resized to 2.5 × 2.5 µm² by averaging signals of multiple neighboring pixels. The final image size was 400 × 400 pixels (i.e. 40 × 40 pixels for each unit image).

 

Fig. 5. Polarization imaging with compound-eye metalens. (a) Optical setup for virtually simulating proposed system by imaging plane where image sensor is assumed to be placed. 1.0 × 1.0 mm² compound-eye metalens that simultaneously creates 10 × 10 images was used. Printed mandrill image covered with four polarizer films was used as target object. White arrows in image indicate transmission axis of films. (b) Measured raw compound-eye image consisting of 10 × 10 unit images. Image size is 400 × 400 pixels. Scale bar: 200 µm. Its enlarged images are also shown. (c) Intensity, AoP, and degree of linear polarization (DoLP) images reconstructed from raw image. Each image consists of 200 × 200 pixels.

Download Full Size | PDF

Figure 5(b) shows the captured raw compound-eye image consisting of 10 × 10 unit images. From the enlarged images [the center and right images in Fig. 5(b)], it is evident that the images were formed while being sorted according to the polarization basis. We then applied the reconstruction algorithm based on an inverse problem to this compound-eye image for recovering the spatial resolution. Although the inverse problem can be solved using several numerical approaches, we used the steepest descent method. Finally, the intensity, AoP, and DoLP images were reconstructed from the obtained S_0-2 in the scene. Figure 5(c) shows the reconstructed images, illustrating the ability to image the polarization information. The intensity image (left) clearly shows the mandrill image behind the polarizer films, but there was no difference between the films. The AoP image (center) instead accurately reveals their axis orientations. The DoLP image (right) also shows that the light transmitted through the films is strongly polarized. Furthermore, the spatial resolution of all images (200 × 200 pixels) dramatically improved compared to that of the raw unit image (40 ×40 pixels). The number of pixels available for raw imaging is now limited to 400 × 400 due to the field of view of the microscope, but the direct integration of metalenses into a sensor mitigates this limitation and further enhances the resolution of the final images. This indicates the potential to create high-resolution polarization images by fully using the sensor pixels while reducing device thickness.

It should be noted that these results were obtained with the reconstruction algorithm based on the system matrix with which the ideal optical functions for the system are assumed. Therefore, the image quality will further improve by introducing a calibrated system matrix that takes into account the metalenses’ point spread functions as well as the variations possibly caused by fabrication errors and incident angles. The polarimetry accuracy can also improve by inserting absorptive separators between the neighboring metalenses, as commonly used in compound-eye architectures to reduce the impact of spatial crosstalk between adjacent unit images [47,48]. In fact, the compound-eye metalens shows a ∼14% polarization crosstalk (the transmission-power-normalized power detected by pixels comprising one unit image when the linear polarization orthogonal to its preferred one is incident), which was measured by imaging a polarizer-covered printed image whose size is comparable to the field-of-view of the system. The crosstalk value includes the effect of the ER (∼5%), so ∼9% may be attributed to overlapping between the neighboring unit images due to the parallax-based image shift as well as imaging light outside the field-of-view. Although it is difficult to completely eliminate the crosstalk because of the finite ER value of the metalens, it is possible to eliminate the crosstalk components due to the image-overlapping by inserting absorptive separators.

We also investigated imaging performance at the other incident wavelengths. In Appendix A4, we present images of a color object captured through metalenses at wavelengths of 450, 520, 635 nm. These images show chromatic aberrations, which can be ascribed to off-axis focusing as well as the intrinsic dispersion of the metalenses [37]. As such, there are limitations to simply extending the design to polarization-color imaging in the current stage; this is in contrast to the conventional filter-based systems with an achromatic imaging lens and typical polarizers operating across broadband wavelengths. To achieve polarization-color imaging with a compound-eye metalens, one can use chromatic-dispersion engineering techniques [35,54] as well as advanced optimization methods [55] in the metalens design. Another possible path is composing a compound eye with an array of metalenses designed at different spectral bands (corresponding to red, green, and blue) and using it combined with a color filter array. This will mitigate chromatic aberrations and relax the bandwidth requirements in the above advanced design.

Although this study focused on polarization imaging to acquire S_0-2 over a scene, the basic concept can be extended to full-Stokes polarimetry imaging. This is based on the fact that dielectric metasurfaces can be designed as a polarization-splitting lens for circularly polarized light as well [37,43]. Therefore, the side-by-side arrangement of the linear- and circular-polarization-splitting metalenses allows us to acquire all the polarization states simultaneously, enabling the fabrication of a filter-free, ultra-thin, full-Stokes polarization camera. Another appealing feature of a compound-eye metalens is that it simultaneously acquires the angle information of the incident from an object as the parallax shift of unit images. When combined with computational techniques [56], this feature enables the distance estimation of objects as well as refocusing functionality without any moving components. Therefore, the proposed system has great potential to capture various types of optical information over a scene with snapshot operation.

5. Imaging sensitivity

Maximizing the light utilization efficiency with a filter-free scheme naturally improves the signal-to-noise ratio (SNR) of raw signals, contributing to the improvement in image SNR (i.e. imaging sensitivity) [46]. Our filter-free system with ∼70% light utilization efficiency therefore has the potential to improve the SNR of polarization images when the errors of the reconstruction algorithm are small enough. To evaluate this, we finally compared the noise-against performance of our system [Fig. 1(b)] with that of a conventional DoFP system based on metal-wire-grid micropolarizers [18] [Fig. 1(a)]. We simulated the image-formation models of the proposed and DoFP systems with the test polarization image (512 × 512 pixels) shown in Fig. 6(a). We used the experimentally measured optical properties for the metalenses (light utilization efficiency: 68.4%, ER: 19.2) and micropolarizers (light utilization efficiency: 32.6%, ER: 58 [18]). We also assumed that the focal length of the metalenses was 500 µm, number of formed unit images was 8 × 8, pixel size of the sensor was 2.0 × 2.0 µm², and object-metalens distance was 55 mm for the proposed system. To simplify the analysis, in both systems, aberrations and blur in optics were excluded from the simulations, and imaging at a single wavelength was assumed. Figures 6(b) and 6(c) show the raw images formed on the sensor with each system.

 

Fig. 6. Imaging sensitivity. (a) Test polarization image composed of four polarization bases. (b, c) Raw captured images with image-formation models of (b) compound-eye metalens system and (c) conventional DoFP system based on micropolarizers. (d–f) Simulated errors of reconstructed images with both system models as function of peak signal-to-noise ratio (PSNR) of raw sensor signals. (d) PSNRs of intensity image, (e) root-mean-square (RMS) errors of AoP, and (f) RMS errors of DoLP. (g, h) Examples of images reconstructed with models of (g) compound-eye metalens system and (h) conventional DoFP system with sensor noise of 24 and 48 dB. Their visualized errors (value differences from original image) within dotted squares are also shown in each inset.

Download Full Size | PDF

We then calculated the errors of the reconstructed polarization images as a function of the noise level of the raw sensor signals. Briefly, random Gaussian noise was first added to the raw sensor signals. We next created intensity, AoP, and DoLP images (256 × 256 pixels for each) by applying the reconstruction algorithm to the noise-added raw signals (the proposed system) or using the raw signals directly (the DoFP system). By calculating the difference from the original image [Fig. 6(a)] for each pixel, we then obtained the average peak signal-to-noise ratios (PSNRs) for the intensity images and average root-mean-square (RMS) errors for the AoP and DoLP images. This simulation was repeated 10 times independently, and the results were finally averaged. Note that, to obtain the difference from the original image, both reconstructed images were resized to the same size as the original image by simply considering 1 × 1 pixel as 2 × 2 pixels.

Figures 6(d), 6(e), and 6(f) respectively show the simulated image PSNRs, AoP RMS errors, and DoLP RMS errors for both systems. Note that, in the region where the sensor noise is negligibly small (∼50 dB), the simulated errors correspond to errors in the image reconstruction method. Examples of their corresponding images and visualized errors are also summarized in Figs. 6(g) and 6(h). The graphs clearly show that our system (red dots) outperformed the DoFP system (black dots) in which sensor noise is dominant (< ∼36 dB). In this regime, the PSNRs of our system were always above those of the DoFP system, and the AoP and DoLP errors were always below those of the DoFP system, demonstrating its advantage in imaging sensitivity under low-light conditions. Such features are also visible in the corresponding images. These features can be ascribed to the higher light utilization efficiency as well as to the fact that the reconstruction errors are small enough. In the region where the sensor noise is low (> ∼36 dB), the DoLP errors of our system were slightly larger than those of the DoFP system [see Fig. 6(f)] although the PSNRs were better and AoP errors were smaller. This may be due to the lower ER of the metalenses than that of the micropolarizers, as the ER directly determines DoLP accuracy [57]. To further analyze this, we explored how the metalens characteristics impact polarimetry accuracy and noise tolerance (see Appendix A5). The results indicate that a higher efficiency leads to improved noise tolerance in all images and a higher ER mainly increases polarimetry accuracy. They also indicate that our system using metalenses with an ER of ∼60 can always exhibit superior imaging performance to that of the DoFP system at all sensor noise levels. These suggest that the performance of our system can be further enhanced by the improvement in metalens quality, e.g., coating the back surface with an anti-reflection layer, using more advanced optimization for the design [55], using the nanopost library with lower phase errors, and improving fabrication.

6. Conclusion

We proposed a polarization imaging system based on a compound-eye metalens and demonstrated its capability for making a high-sensitivity, ultra-thin polarization camera. The compound-eye metalens enables polarization imaging without moving parts, cascaded optical elements, and a specially patterned sensor array, while maximizing the light utilization efficiency and reducing device size. This could pave the way toward the widespread adoption of polarization imaging. The proposed system also has the potential to simultaneously acquire various types of optical information, not only full-Stokes polarizations but also incident colors and angles. The advantages of these features will be further enhanced when combined with compressive sensing/imaging techniques [58,59], providing a new avenue in imaging technology. We envision that imaging systems with compound-eye metasurface optics will find widespread use in various research and technology fields such as biology, remote sensing, and autonomous vehicles.

Appendix

A1. Optical properties of silicon nitride nanoposts

Figure 7 shows the simulated phase shifts (φ_x and φ_y) and intensity transmittances (T_x and T_y) of the silicon nitride (SiN) nanoposts for x- and y-polarized light as functions of nanopost widths w₁ and w₂. We carried out the simulations under normal incidence at the 520-nm wavelength. We also used the dielectric constant of SiN obtained by linear interpolation of the experimental data. The white dotted squares in the maps indicate the geometry-parameter range used for the metalens design presented in the main text. Using these data, we then found the optimal w₁ and w₂ in a point-by-point manner, which minimize the mean squared error:

(3)$$E = \frac{1}{2}\left( {{{\left|{\sqrt {{T_x}} {e^{i{\varphi_x}}} - {e^{i{\varphi_{xi}}}}} \right|}^2} + {{\left|{\sqrt {{T_y}} {e^{i{\varphi_y}}} - {e^{i{\varphi_{yi}}}}} \right|}^2}} \right),$$

where φ_xi and φ_yi indicate a pair of target phases for x- and y-polarized light at a point. It is worth noting that using nanoposts with a full range of width parameters is promising to further match the phase profiles to the ideal ones. The use of thicker nanoposts is also promising for this purpose. However, these approaches require more delicate fabrication processes that can support even finer nanostructures with a higher aspect ratio; they would result in a trade-off between performance and ease of fabrication. As shown in the main text, our designs that exclude tiny geometries still exhibit sufficient performance for polarization imaging.

 

Fig. 7. Transmission properties of silicon nitride (SiN) nanopost arrays on quartz substrate. (a, b) Phase shifts and intensity transmittances as functions of nanopost widths w₁ and w₂ under (a) x- and (b) y-polarized light illumination at 520-nm wavelength. White dotted squares indicate geometry-parameter range of w₁, w₂ = 120–280 nm.

Download Full Size | PDF

A2. Optical setup for metalens characterizations

Figure 8(a) shows a schematic of the optical setup for measuring the intensity profiles of the light transmitted through the metalenses at the focal plane under plane wave illumination. The setup consists of a custom-built optical microscope. Light from a supercontinuum laser is first collimated and expanded to create virtual plane light illumination. The light is then filtered with a bandpass filter (520 ± 5 nm) and a linear polarizer and illuminates the metalenses. The incident polarization state is controlled by rotating the polarizer. The light transmitted through the metalenses is imaged with a 50× objective lens with a numerical aperture (NA) of 0.8 and finally captured using a monochrome image sensor with a linear response.

 

Fig. 8. Optical setups for metalens characterizations. (a) Schematic illustration of optical setup used for characterizing polarization-dependent focusing of metalenses. (b) Schematic illustration of optical setup used for characterizing polarization-dependent imaging of metalenses.

Download Full Size | PDF

Figure 8(b) shows a schematic of the optical setup for capturing the images formed through the metalenses while changing the incident polarization state. This setup is almost the same as that above for focusing characterization, but the incidence side is completely different. Light from a halogen lamp illuminates a printed mandrill image covered with a linear polarizer from behind. The polarized light transmitted through it is then incident on the metalenses. The images formed by the metalenses at their focal plane are finally captured through the optical microscope. The polarizer covering the mandrill image is used to create different linear polarization states. A bandpass filter (520 ± 5 nm) is inserted into the microscope to characterize the metalenses at the designed wavelength.

A3. Simulated potential performance of dielectric metesurfeces for polarimetry

Figure 9(a) shows the schematic of polarization splitting from the designed polarization splitter. The incident light on the splitter is deflected into +1st-order diffracted light or -1st-order diffracted light according to the polarization. Figure 9(b) shows the designed SiN nanopost patterns and their phase profiles for the polarization splitting. The patterns were designed using SiN nanoposts with nanopost widths of w₁, w₂ = 80–350 nm or w₁, w₂ = 120–280 nm by referring the simulated transmission properties shown in Fig. 7 to match the ideal phase profile [shown as the dashed lines in Fig. 9(b)] at a wavelength of 520 nm. The diffraction period is 5.2 µm. This corresponds to the deflection angle θ_def of 5.74° that is comparable to the off-axis angle of the fabricated metalenses. Their performance was simulated using a rigorous coupled wave analysis. Figure 9(c) shows the simulated diffraction efficiencies for the two designs under plane light illumination (520 nm wavelength) as a function of incident AoP. From the sinusoidal plots, an extinction ratio of ∼97.3 (∼22.1) was obtained for the design using the nanoposts with w₁, w₂ = 80–350 nm (w₁, w₂ = 120–280 nm).

 

Fig. 9. Simulated potential performance of dielectric metesurfeces for polarimetry (a) Schematic of polarization splitting from polarization splitter (b) Designs of polarization splitter using SiN nanoposts with nanopost widths of w₁, w₂ = 80–350 nm or w₁, w₂ = 120–280 nm. (c) Diffraction efficiencies for two designs under plane light illumination (520 nm wavelength) as function of incident angle of polarization (AoP).

Download Full Size | PDF

A4. Chromatic aberration of metalenses

Figures 10(a) shows a schematic of the optical setup for characterizing the imaging performance at three wavelengths of 450, 520, 635 nm. This setup is almost the same as that shown in Fig. 8(b), except the target object. A printed color mandrill image covered with four different linear polarizers was used as a target object. In the characterization, we used the metalenses designed for a 100 × 200 µm² aperture, 500-µm focal length, and 520-nm operating wavelength. In addition, the bandpass filters were alternatively inserted into the setup to extract the images corresponding to these three wavelengths.

 

Fig. 10. Chromatic aberration of metalenses. (a) Optical setup for metalens characterizations. Bandpass filters are alternatively inserted into setup to extract images corresponding to wavelengths of 450, 520, and 635 nm. (b) Images captured at these wavelengths.

Download Full Size | PDF

Figure 10(b) shows the captured images on the focal plane (0.5 mm from the metalenses) at different wavelengths. Only the image at the 520-nm wavelength is clear while the others are out of focus, showing the chromatic aberration effects of the metalenses. One can also selectively focus onto other wavelengths by adjusting the image plane of the objective lens.

A5. Impact of metalens characteristics on polarization imaging

Figures 11(a)–11(c) show the impact of the extinction ratio (ER) of the metalenses on the accuracy and noise tolerance of polarization imaging. We simulated the peak signal-to-noise ratios (PSNRs) of the intensity image, root-mean-square (RMS) errors of the angle of polarization (AoP), and RMS errors of the degree of linear polarization (DoLP) as a function of the PSNR of the raw sensor signals while varying the ER. The simulations were conducted with the same procedure as described in the main text. The light utilization efficiency of the metalenses was fixed to 68.4% (experimentally measured value) for all plots. Increasing the ER mainly leads to improvement in DoLP accuracy. Similarly, we investigated the impact of the light utilization efficiency of the metalenses, as summarized in Figs. 11(d)–11(f). The ER of the metalenses was fixed to 19.2 (experimentally measured value) for all plots. The graphs show that increasing the efficiency improves noise-tolerance performance.

 

Fig. 11. Impact of metalens characteristics on polarimetry accuracy and noise tolerance. (a–c) Impact of extinction ratio (ER) of metalenses. (a) Peak signal-to-noise ratios (PSNRs) of intensity image, (b) root-mean-square (RMS) errors of AoP, and (c) RMS errors of degree of linear polarization (DoLP) as function of PSNR of raw sensor signals. Light utilization efficiency of metalenses was 68.4% for all plots. (d–f) Impact of light utilization efficiency of metalenses. (d) PSNRs of intensity image, (e) RMS errors of AoP, and (f) RMS errors of DoLP as function of PSNR of raw sensor signals. ER of metalenses was 19.2 for all plots. Errors with division-of-focal-plane (DoFP) polarization imaging system are also plotted as black dots in all graphs for reference.

Download Full Size | PDF

Acknowledgments

We thank Naru Nemoto (NTT Device Technology Laboratories) for helpful discussions.

Disclosures

The authors declare no conflicts of interest.

References

1. J. S. Tyo, D. L. Goldstein, D. B. Chenault, and J. A. Shaw, “Review of passive imaging polarimetry for remote sensing applications,” Appl. Opt. 45(22), 5453–5469 (2006). [CrossRef]  

2. N. M. Garcia, I. De Erausquin, C. Edmiston, and V. Gruev, “Surface normal reconstruction using circularly polarized light,” Opt. Express 23(11), 14391–14406 (2015). [CrossRef]  

3. L. B. Wolff and T. E. Boult, “Constraining object features using a polarization reflectance model,” IEEE Trans. Pattern Anal. Machine Intell. 13(7), 635–657 (1991). [CrossRef]  

4. J. S. Tyo, M. P. Rowe, E. N. Pugh, and N. Engheta, “Target detection in optically scattering media by polarization-difference imaging,” Appl. Opt. 35(11), 1855–1870 (1996). [CrossRef]  

5. S. G. Demos and R. R. Alfano, “Optical polarization imaging,” Appl. Opt. 36(1), 150–155 (1997). [CrossRef]  

6. Y. Y. Schechner, S. G. Narasimhan, and S. K. Nayar, “Polarization-based vision through haze,” Appl. Opt. 42(3), 511–525 (2003). [CrossRef]  

7. S. Tominaga and A. Kimachi, “Polarization imaging for material classification,” Opt. Eng. 47(12), 123201 (2008). [CrossRef]  

8. E. Salomatina-Motts, V. A. Neel, and A. N. Yaroslavskaya, “Multimodal Polarization System for Imaging Skin Cancer,” Opt. Spectrosc. 107(6), 884–890 (2009). [CrossRef]  

9. T. York, L. Kahan, S. P. Lake, and V. Gruev, “Real-time high-resolution measurement of collagen alignment in dynamically loaded soft tissue,” J. Biomed. Opt. 19(6), 066011 (2014). [CrossRef]  

10. T. Charanya, T. York, S. Bloch, G. Sudlow, K. Liang, M. Garcia, W. J. Akers, D. Rubin, V. Gruev, and S. Achilefu, “Trimodal color-fluorescence- polarization endoscopy aided by a tumor selective molecular probe accurately detects flat lesions in colitis-associated cancer,” J. Biomed. Opt. 19(12), 126002 (2014). [CrossRef]  

11. D. Wang, H. Liang, H. Zhu, and S. Zhang, “A Bionic Camera-Based Polarization Navigation Sensor,” Sensors 14(7), 13006–13023 (2014). [CrossRef]  

12. S. B. Powell, R. Garnett, J. Marshall, C. Rizk, and V. Gruev, “Bioinspired polarization vision enables underwater geolocalization,” Sci. Adv. 4(4), eaao6841 (2018). [CrossRef]  

13. R. Walraven, “Polarization imagery,” Opt. Eng. 20(1), 14–18 (1981). [CrossRef]  

14. W. G. Egan, W. R. Johnson, and V. S. Whitehead, “Terrestrial polarization imagery obtained from the Space Shuttle: characterization and interpretation,” Appl. Opt. 30(4), 435–442 (1991). [CrossRef]  

15. C. A. Farlow, D. B. Chenault, J. L. Pezzaniti, K. D. Spradley, and M. G. Gulley, “Imaging polarimeter development and applications,” Proc. SPIE 4481, 118 (2002). [CrossRef]  

16. J. L. Pezzaniti and D. B. Chenault, “A division of aperture MWIR imaging polarimeter,” Proc. SPIE 5888, 58880V (2005). [CrossRef]  

17. A. G. Andreou and Z. K. Kalayjian, “Polarization Imaging: Principles and Integrated Polarimeters,” IEEE Sens. J. 2(6), 566–576 (2002). [CrossRef]  

18. V. Gruev, R. Perkins, and T. York, “CCD polarization imaging sensor with aluminum nanowire optical filters,” Opt. Express 18(18), 19087–19094 (2010). [CrossRef]  

19. M. Garcia, C. Edmiston, R. Marinov, A. Vail, and V. Gruev, “Bio-inspired color-polarization imager for real-time in situ imaging,” Optica 4(10), 1263–1271 (2017). [CrossRef]  

20. M. Garcia, T. Davis, S. Blair, N. Cui, and V. Gruev, “Bioinspired polarization imager with high dynamic range,” Optica 5(10), 1240–1246 (2018). [CrossRef]  

21. N. I. Zheludev and Y. S. Kivshar, “From metamaterials to metadevices,” Nat. Mater. 11(11), 917–924 (2012). [CrossRef]  

22. N. Yu and F. Capasso, “Flat optics with designer metasurfaces,” Nat. Mater. 13(2), 139–150 (2014). [CrossRef]  

23. A. I. Kuznetsov, A. E. Miroshnichenko, M. L. Brongersma, Y. S. Kivshar, and B. Luk’yanchuk, “Optically resonant dielectric nanostructures,” Science 354(6314), aag2472 (2016). [CrossRef]  

24. M. Khorasaninejad and F. Capasso, “Metalenses: Versatile multifunctional photonic components,” Science 358(6367), eaam8100 (2017). [CrossRef]  

25. S. M. Kamali, E. Arbabi, A. Arbabi, and A. Faraon, “A review of dielectric optical metasurfaces for wavefront control,” Nanophotonics 7(6), 1041–1068 (2018). [CrossRef]  

26. M. Song, D. Wang, S. Peana, S. Choudhury, P. Nyga, Z. A. Kudyshev, H. Yu, A. Boltasseva, V. M. Shalaev, and A. V. Kildishev, “Colors with plasmonic nanostructures: A full- spectrum review,” Appl. Phys. Rev. 6(4), 041308 (2019). [CrossRef]  

27. P. Lalanne, S. Astilean, and P. Chavel, “Design and fabrication of blazed binary diffractive elements with sampling periods smaller than the structural cutoff,” J. Opt. Soc. Am. A 16(5), 1143–1156 (1999). [CrossRef]  

28. Z. Bomzon, G. Biener, V. Kleiner, and E. Hasman, “Space-variant Pancharatnam – Berry phase optical elements with computer-generated subwavelength gratings,” Opt. Lett. 27(13), 1141–1143 (2002). [CrossRef]  

29. D. Fattal, J. Li, Z. Peng, M. Fiorentino, and R. G. Beausoleil, “Flat dielectric grating reflectors with focusing abilities,” Nat. Photonics 4(7), 466–470 (2010). [CrossRef]  

30. K. Kumar, H. Duan, R. S. Hegde, S. C. W. Koh, J. N. Wei, and J. K. W. Yang, “Printing colour at the optical diffraction limit,” Nat. Nanotechnol. 7(9), 557–561 (2012). [CrossRef]  

31. S. Yokogawa, S. P. Burgos, and H. A. Atwater, “Plasmonic Color Filters for CMOS Image Sensor Applications,” Nano Lett. 12(8), 4349–4354 (2012). [CrossRef]  

32. D. Lin, P. Fan, E. Hasman, and M. L. Brongersma, “Dielectric gradient metasurface optical elements,” Science 345(6194), 298–302 (2014). [CrossRef]  

33. A. Arbabi, Y. Horie, M. Bagheri, and A. Faraon, “Dielectric metasurfaces for complete control of phase and polarization with subwavelength spatial resolution and high transmission,” Nat. Nanotechnol. 10(11), 937–943 (2015). [CrossRef]  

34. M. Miyata, H. Hatada, and J. Takahara, “Full-Color Subwavelength Printing with Gap-Plasmonic Optical Antennas,” Nano Lett. 16(5), 3166–3172 (2016). [CrossRef]  

35. W. T. Chen, A. Y. Zhu, V. Sanjeev, M. Khorasaninejad, Z. Shi, E. Lee, and F. Capasso, “A broadband achromatic metalens for focusing and imaging in the visible,” Nat. Nanotechnol. 13(3), 220–226 (2018). [CrossRef]  

36. M. Miyata, M. Nakajima, and T. Hashimoto, “Impedance-matched dielectric metasurfaces for non-discrete wavefront engineering,” J. Appl. Phys. 125(10), 103106 (2019). [CrossRef]  

37. M. Khorasaninejad, W. T. Chen, A. Y. Zhu, J. Oh, R. C. Devlin, D. Rousso, and F. Capasso, “Multispectral Chiral Imaging with a Metalens,” Nano Lett. 16(7), 4595–4600 (2016). [CrossRef]  

38. A. Arbabi, E. Arbabi, S. M. Kamali, Y. Horie, S. Han, and A. Faraon, “Miniature optical planar camera based on a wide-angle metasurface doublet corrected for monochromatic aberrations,” Nat. Commun. 7(1), 13682 (2016). [CrossRef]  

39. B. H. Chen, P. C. Wu, V.-C. Su, Y.-C. Lai, C. H. Chu, I. C. Lee, J.-W. Chen, Y. H. Chen, Y.-C. Lan, C.-H. Kuan, and D. P. Tsai, “GaN Metalens for Pixel-Level Full-Color Routing at Visible Light,” Nano Lett. 17(10), 6345–6352 (2017). [CrossRef]  

40. S. Colburn, A. Zhan, and A. Majumdar, “Metasurface optics for full-color computational imaging,” Sci. Adv. 4(2), eaar2114 (2018). [CrossRef]  

41. Z. Yang, Z. Wang, Y. Wang, X. Feng, M. Zhao, Z. Wan, L. Zhu, J. Liu, Y. Huang, J. Xia, and M. Wegener, “Generalized Hartmann-Shack array of dielectric metalens sub-arrays for polarimetric beam profiling,” Nat. Commun. 9(1), 4607 (2018). [CrossRef]  

42. H. Kwon, E. Arbabi, S. M. Kamali, S. Faraji-Dana, and A. Faraon, “Computational complex optical field imaging using a designed metasurface diffuser,” Optica 5(8), 924–931 (2018). [CrossRef]  

43. E. Arbabi, S. M. Kamali, A. Arbabi, and A. Faraon, “Full-Stokes Imaging Polarimetry Using Dielectric Metasurfaces,” ACS Photonics 5(8), 3132–3140 (2018). [CrossRef]  

44. N. A. Rubin, G. D’Aversa, P. Chevalier, Z. Shi, W. T. Chen, and F. Capasso, “Matrix Fourier optics enables a compact full-Stokes polarization camera,” Science 365(6448), eaax1839 (2019). [CrossRef]  

45. R. J. Lin, V. Su, S. Wang, M. K. Chen, T. L. Chung, Y. H. Chen, H. Y. Kuo, J. Chen, J. Chen, Y. Huang, J. Wang, C. H. Chu, P. C. Wu, T. Li, Z. Wang, S. Zhu, and D. P. Tsai, “Achromatic metalens array for full-colour light-field imaging,” Nat. Nanotechnol. 14(3), 227–231 (2019). [CrossRef]  

46. M. Miyata, M. Nakajima, and T. Hashimoto, “High-Sensitivity Color Imaging Using Pixel-Scale Color Splitters Based on Dielectric Metasurfaces,” ACS Photonics 6(6), 1442–1450 (2019). [CrossRef]  

47. J. Tanida, T. Kumagai, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, and Y. Ichioka, “Thin observation module by bound optics (TOMBO): concept and experimental verification,” Appl. Opt. 40(11), 1806–1813 (2001). [CrossRef]  

48. A. Brückner, J. Duparré, R. Leitel, P. Dannberg, A. Bräuer, and A. Tünnermann, “Thin wafer-level camera lenses inspired by insect compound eyes,” Opt. Express 18(24), 24379–24394 (2010). [CrossRef]  

49. Y. M. Song, Y. Xie, V. Malyarchuk, J. Xiao, I. Jung, K. Choi, Z. Liu, H. Park, C. Lu, R.-H. Kim, R. Li, K. B. Crozier, Y. Huang, and J. A. Rogers, “Digital cameras with designs inspired by the arthropod eye,” Nature 497(7447), 95–99 (2013). [CrossRef]  

50. S. Colburn, A. Zhan, E. Bayati, J. Whitehead, A. Ryou, L. Huang, and A. Majumdar, “Broadband transparent and CMOS-compatible flat optics with silicon nitride metasurfaces,” Opt. Mater. Express 8(8), 2330–2344 (2018). [CrossRef]  

51. K. Sasagawa, S. Shishido, K. Ando, H. Matsuoka, T. Noda, T. Tokuda, K. Kakiuchi, and J. Ohta, “Image sensor pixel with on-chip high extinction ratio polarizer based on 65-nm standard CMOS technology,” Opt. Express 21(9), 11132–11140 (2013). [CrossRef]  

52. W. Hsu, G. Myhre, K. Balakrishnan, N. Brock, M. Ibn-elhaj, and S. Pau, “Full-Stokes imaging polarimeter using an array of elliptical polarizer,” Opt. Express 22(3), 3063–3074 (2014). [CrossRef]  

53. Z. Wang, J. Chu, Q. Wang, and R. Zhang, “Single-Layer Nanowire Polarizer Integrated With Photodetector and Its Application for Polarization Navigation,” IEEE Sens. J. 16(17), 6579–6585 (2016). [CrossRef]  

54. E. Arbabi, A. Arbabi, S. M. Kamali, Y. Horie, and A. Faraon, “Controlling the sign of chromatic dispersion in diffractive optics with dielectric metasurfaces,” Optica 4(6), 625–632 (2017). [CrossRef]  

55. D. Sell, J. Yang, S. Doshay, R. Yang, and J. A. Fan, “Large-Angle, Multifunctional Metagratings Based on Freeform Multimode Geometries,” Nano Lett. 17(6), 3752–3757 (2017). [CrossRef]  

56. R. Horisaki, S. Irie, Y. Ogura, and J. Tanida, “Three-Dimensional Information Acquisition Using a Compound Imaging System,” Opt. Rev. 14(5), 347–350 (2007). [CrossRef]  

57. T. York and V. Gruev, “Characterization of a visible spectrum division-of-focal-plane polarimeter,” Appl. Opt. 51(22), 5392–5400 (2012). [CrossRef]  

58. R. Horisaki, K. Choi, J. Hahn, J. Tanida, and D. J. Brady, “Generalized sampling using a compound-eye imaging system for multi-dimensional object acquisition Abstract:,” Opt. Express 18(18), 19367–19378 (2010). [CrossRef]  

59. R. Horisaki, X. Xiao, J. Tanida, and B. Javidi, “Feasibility study for compressive multi-dimensional integral imaging,” Opt. Express 21(4), 4263–4279 (2013). [CrossRef]